Skip to content

Commit 3399294

Browse files
committed
Mention TRT LLM backend supports request cancellation (#6585)
1 parent a984bf4 commit 3399294

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

docs/user_guide/request_cancellation.md

+1
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ at various points. However, once a request has been given to the backend
8282
for execution, it is up to the individual backends to detect and handle
8383
request termination.
8484
Currently, the following backends support early termination:
85+
- [TensorRT-LLM backend](https://github.com/triton-inference-server/tensorrtllm_backend)
8586
- [vLLM backend](https://github.com/triton-inference-server/vllm_backend)
8687
- [python backend](https://github.com/triton-inference-server/python_backend)
8788

0 commit comments

Comments
 (0)