Skip to content

Commit 441d930

Browse files
authored
Mention TRT LLM backend supports request cancellation (#6585)
1 parent ee68c57 commit 441d930

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

docs/user_guide/request_cancellation.md

+1
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ at various points. However, once a request has been given to the backend
8282
for execution, it is up to the individual backends to detect and handle
8383
request termination.
8484
Currently, the following backends support early termination:
85+
- [TensorRT-LLM backend](https://github.com/triton-inference-server/tensorrtllm_backend)
8586
- [vLLM backend](https://github.com/triton-inference-server/vllm_backend)
8687
- [python backend](https://github.com/triton-inference-server/python_backend)
8788

0 commit comments

Comments
 (0)