@@ -75,18 +75,22 @@ pip install dlinfer-ascend
75
75
76
76
## LMDeploy
77
77
78
- | | 华为Atlas 800T A2(bf16, w4a16) | 沐曦C500 | 寒武纪云端智能加速卡(开发中) |
79
- | --- | --- | --- | --- |
80
- | InternLM2.5-7B/20B | √,√ | √ | |
81
- | InternLM2-7B/20B | √,√ | √ | |
82
- | InternVL2-2B | √,√ | √ | |
83
- | InternVL1-5 | √,√ | √ | |
84
- | Llama3-8B | √,√ | √ | |
85
- | Mixtral8x7B | √,X | √ | |
86
- | Qwen2-7B | √,X | √ | |
87
- | Qwen2-57B-A14B | √,X | √ | |
88
- | CogVLM | √,X | √ | |
89
- | CogVLM2 | √,X | √ | |
78
+ | | | 华为Atlas 800T A2 | | 沐曦C500 | 寒武纪云端智能加速卡(开发中) |
79
+ | --- | --- | --- | --- | --- | --- |
80
+ | | bf16(eager) | w4a16(eager) | bf16(graph) | | |
81
+ | InternLM2.5-7B/20B | √ | √ | √ | √ | |
82
+ | InternLM2-7B/20B | √ | √ | √ | √ | |
83
+ | InternVL2-2B | √ | √ | √ | √ | |
84
+ | InternVL1-5 | √ | √ | - | √ | |
85
+ | Llama3(.1)-8B | √ | √ | √ | √ | |
86
+ | Mixtral8x7B | √ | X | √ | √ | |
87
+ | Qwen2(.5)-7B | √ | X | √ | √ | |
88
+ | Qwen2-57B-A14B | √ | X | - | √ | |
89
+ | CogVLM | √ | X | - | √ | |
90
+ | CogVLM2 | √ | X | - | √ | |
91
+ | glm-4v-9b | √ | - | - | - | |
92
+
93
+ ‘√’代表测试通过,‘X’代表不支持,‘-’代表未测试
90
94
91
95
### 使用LMDeploy
92
96
@@ -113,7 +117,7 @@ if __name__ == "__main__":
113
117
```
114
118
115
119
> [ !TIP]
116
- > 图模式已经支持了Atlas 800T A2。目前,单卡下的LLaMa3-8B/LLaMa2-7B/Qwen2-7B已经通过测试。
120
+ > 图模式已经支持了Atlas 800T A2。
117
121
> 用户可以在离线模式下设定` PytorchEngineConfig ` 中的` eager_mode=False ` 来开启图模式,或者设定` eager_mode=True ` 来关闭图模式。
118
122
> 在线模式下默认开启图模式,请添加` --eager-mode ` 来关闭图模式。
119
123
> (启动图模式需要事先` source /usr/local/Ascend/nnal/atb/set_env.sh ` )
0 commit comments