Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Feb 18, 2025.

This update includes:

• Features
  • Added INT4-AWQ support for MoE models. Refer to the “AWQ Quantization” section in examples/mixtral/README.md. (Fail to build int4_awq on Mixtral 8x7b #1580, How to build int4_gptq on Mixtral 8x7b #1581)
• Bug fixes
  • Fixed misused length argument of PluginField. Thanks to the contribution from @jl749 in fix: gptattentionplugin onnxparser compatability #2712. This also fixes GPTAttentionPlugin missing declaration of fields #2685.
• Performance
  • Improved LoRA performance for multiple requests with different task ids.
• Infra
  • Updated base Rocky Linux 8 docker image to 12.8.0-devel-rockylinux8.
  • Updated upstream PyTorch dependency to 2.6.0.

Thanks,
The TensorRT-LLM Engineering Team