Skip to content

Commit 2010508

Browse files
qinxuyeqinxuye
authored
DOC: small fixes for doc (#3294)
1 parent e96ac6b commit 2010508

File tree

10 files changed

+75
-41
lines changed

10 files changed

+75
-41
lines changed

‎doc/source/getting_started/installation.rst

+2
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ Currently, supported models include:
6161
- ``QwQ-32B-Preview``, ``QwQ-32B``
6262
- ``marco-o1``
6363
- ``fin-r1``
64+
- ``seallms-v3``
65+
- ``skywork-or1-preview``
6466
- ``gemma-it``, ``gemma-2-it``, ``gemma-3-1b-it``
6567
- ``orion-chat``, ``orion-chat-rag``
6668
- ``c4ai-command-r-v01``

‎doc/source/locale/zh_CN/LC_MESSAGES/models/virtualenv.po

+33-20
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ msgid ""
88
msgstr ""
99
"Project-Id-Version: Xinference \n"
1010
"Report-Msgid-Bugs-To: \n"
11-
"POT-Creation-Date: 2025-04-19 00:37+0800\n"
11+
"POT-Creation-Date: 2025-04-19 14:42+0000\n"
1212
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
1313
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
1414
"Language: zh_CN\n"
@@ -17,7 +17,7 @@ msgstr ""
1717
"MIME-Version: 1.0\n"
1818
"Content-Type: text/plain; charset=utf-8\n"
1919
"Content-Transfer-Encoding: 8bit\n"
20-
"Generated-By: Babel 2.14.0\n"
20+
"Generated-By: Babel 2.16.0\n"
2121

2222
#: ../../source/models/virtualenv.rst:5
2323
msgid "Model Virtual Environments"
@@ -41,9 +41,10 @@ msgid ""
4141
"latest version of ``transformers``. This version mismatch leads to "
4242
"dependency conflicts."
4343
msgstr ""
44-
"一些模型在发布后不再维护，其依赖的库版本也保持在较旧的状态。例如，``GOT-OCR2`` 模型仍依赖于 "
45-
"``transformers`` 4.37.2。如果将该库升级为新版本，模型将无法正常运行；"
46-
"而许多新模型又需要最新版本的 ``transformers``。这种版本差异会导致依赖冲突。"
44+
"一些模型在发布后不再维护，其依赖的库版本也保持在较旧的状态。例如，``GOT-"
45+
"OCR2`` 模型仍依赖于 ``transformers`` 4.37.2。如果将该库升级为新版本，模型"
46+
"将无法正常运行；而许多新模型又需要最新版本的 ``transformers``。这种版本"
47+
"差异会导致依赖冲突。"
4748

4849
#: ../../source/models/virtualenv.rst:20
4950
msgid "Solution"
@@ -56,64 +57,76 @@ msgid ""
5657
msgstr "为了解决这个问题，我们引入了 **模型虚拟空间** 功能。"
5758

5859
#: ../../source/models/virtualenv.rst:24
60+
msgid "Install requirements for this functionality via"
61+
msgstr "通过以下命令安装该功能所需的依赖"
62+
63+
#: ../../source/models/virtualenv.rst:33
5964
msgid ""
6065
"Enable by setting environment variable "
6166
"``XINFERENCE_ENABLE_VIRTUAL_ENV=1``."
6267
msgstr "通过设置环境变量 ``XINFERENCE_ENABLE_VIRTUAL_ENV=1`` 启用该功能。"
6368

64-
#: ../../source/models/virtualenv.rst:26
69+
#: ../../source/models/virtualenv.rst:35
6570
msgid "Example usage:"
6671
msgstr "使用示例："
6772

68-
#: ../../source/models/virtualenv.rst:38
73+
#: ../../source/models/virtualenv.rst:47
6974
msgid "This feature requires internet access or a self-hosted PyPI mirror."
7075
msgstr "该功能需要联网，或使用自建的 PyPI 镜像服务。"
7176

72-
#: ../../source/models/virtualenv.rst:42
77+
#: ../../source/models/virtualenv.rst:49
78+
msgid "Xinference will by default inherit the config for current pip."
79+
msgstr "Xinference 默认会继承当前 pip 的配置。"
80+
81+
#: ../../source/models/virtualenv.rst:53
7382
msgid ""
7483
"The model virtual environment feature is disabled by default (i.e., "
7584
"XINFERENCE_ENABLE_VIRTUAL_ENV is set to 0)."
7685
msgstr ""
77-
"模型虚拟空间功能默认处于关闭状态（即 ``XINFERENCE_ENABLE_VIRTUAL_ENV`` 的默认值为 0）。"
86+
"模型虚拟空间功能默认处于关闭状态（即 ``XINFERENCE_ENABLE_VIRTUAL_ENV`` 的"
87+
"默认值为 0）。"
7888

79-
#: ../../source/models/virtualenv.rst:44
89+
#: ../../source/models/virtualenv.rst:55
8090
msgid "It will be enabled by default starting from Xinference v2.0.0."
8191
msgstr "该功能将在 Xinference v2.0.0 起默认开启。"
8292

83-
#: ../../source/models/virtualenv.rst:46
93+
#: ../../source/models/virtualenv.rst:57
8494
msgid ""
8595
"When enabled, Xinference will automatically create a dedicated virtual "
8696
"environment for each model when it is loaded, and install its specific "
8797
"dependencies there. This prevents dependency conflicts between models, "
8898
"allowing them to run in isolation without affecting one another."
8999
msgstr ""
90-
"启用该功能后，Xinference 会在加载模型时自动为其创建专属的虚拟环境，并在其中安装对应依赖。"
91-
"这可避免模型之间的依赖冲突，确保各模型在相互隔离的环境中独立运行。"
100+
"启用该功能后，Xinference 会在加载模型时自动为其创建专属的虚拟环境，并在"
101+
"其中安装对应依赖。这可避免模型之间的依赖冲突，确保各模型在相互隔离的环境"
102+
"中独立运行。"
92103

93-
#: ../../source/models/virtualenv.rst:51
104+
#: ../../source/models/virtualenv.rst:62
94105
msgid "Supported Models"
95106
msgstr "支持的模型"
96107

97-
#: ../../source/models/virtualenv.rst:53
108+
#: ../../source/models/virtualenv.rst:64
98109
msgid "Currently, this feature supports the following models:"
99110
msgstr "当前，该功能支持以下模型："
100111

101-
#: ../../source/models/virtualenv.rst:55
112+
#: ../../source/models/virtualenv.rst:66
102113
msgid ":ref:`GOT-OCR2 <models_builtin_got-ocr2_0>`"
103114
msgstr ":ref:`GOT-OCR2 <models_builtin_got-ocr2_0>`"
104115

105-
#: ../../source/models/virtualenv.rst:56
116+
#: ../../source/models/virtualenv.rst:67
106117
msgid ":ref:`Qwen2.5-omni <models_llm_qwen2.5-omni>`"
107118
msgstr ":ref:`Qwen2.5-omni <models_llm_qwen2.5-omni>`"
108119

109-
#: ../../source/models/virtualenv.rst:59
120+
#: ../../source/models/virtualenv.rst:70
110121
msgid "Storage Location"
111122
msgstr "存储位置"
112123

113-
#: ../../source/models/virtualenv.rst:61
124+
#: ../../source/models/virtualenv.rst:72
114125
msgid ""
115126
"By default, the model’s virtual environment is stored under path: :ref:`"
116127
"XINFERENCE_HOME <environments_xinference_home>` / virtualenv / {model_"
117128
"name}"
118129
msgstr ""
119-
"默认情况下，模型的虚拟环境存储在以下路径：:ref:`XINFERENCE_HOME <environments_xinference_home>` / virtualenv / {model_name}"
130+
"默认情况下，模型的虚拟环境存储在以下路径：:ref:`XINFERENCE_HOME <"
131+
"environments_xinference_home>` / virtualenv / {model_name}"
132+

‎doc/source/models/builtin/llm/internvl3.rst

+14-14
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Model Spec 1 (pytorch, 1 Billion)
2020
- **Model Format:** pytorch
2121
- **Model Size (in billions):** 1
2222
- **Quantizations:** 8-bit, none
23-
- **Engines**: Transformers
23+
- **Engines**: vLLM, Transformers
2424
- **Model ID:** OpenGVLab/InternVL3-1B
2525
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-1B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-1B>`__
2626

@@ -36,7 +36,7 @@ Model Spec 2 (awq, 1 Billion)
3636
- **Model Format:** awq
3737
- **Model Size (in billions):** 1
3838
- **Quantizations:** Int4
39-
- **Engines**: Transformers
39+
- **Engines**: vLLM, Transformers
4040
- **Model ID:** OpenGVLab/InternVL3-1B-AWQ
4141
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-1B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-1B-AWQ>`__
4242

@@ -52,7 +52,7 @@ Model Spec 3 (pytorch, 2 Billion)
5252
- **Model Format:** pytorch
5353
- **Model Size (in billions):** 2
5454
- **Quantizations:** 8-bit, none
55-
- **Engines**: Transformers
55+
- **Engines**: vLLM, Transformers
5656
- **Model ID:** OpenGVLab/InternVL3-2B
5757
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-2B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-2B>`__
5858

@@ -68,7 +68,7 @@ Model Spec 4 (awq, 2 Billion)
6868
- **Model Format:** awq
6969
- **Model Size (in billions):** 2
7070
- **Quantizations:** Int4
71-
- **Engines**: Transformers
71+
- **Engines**: vLLM, Transformers
7272
- **Model ID:** OpenGVLab/InternVL3-2B-AWQ
7373
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-2B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-2B-AWQ>`__
7474

@@ -84,7 +84,7 @@ Model Spec 5 (pytorch, 8 Billion)
8484
- **Model Format:** pytorch
8585
- **Model Size (in billions):** 8
8686
- **Quantizations:** 8-bit, none
87-
- **Engines**: Transformers
87+
- **Engines**: vLLM, Transformers
8888
- **Model ID:** OpenGVLab/InternVL3-8B
8989
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-8B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-8B>`__
9090

@@ -100,7 +100,7 @@ Model Spec 6 (awq, 8 Billion)
100100
- **Model Format:** awq
101101
- **Model Size (in billions):** 8
102102
- **Quantizations:** Int4
103-
- **Engines**: Transformers
103+
- **Engines**: vLLM, Transformers
104104
- **Model ID:** OpenGVLab/InternVL3-8B-AWQ
105105
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-8B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-8B-AWQ>`__
106106

@@ -116,7 +116,7 @@ Model Spec 7 (pytorch, 9 Billion)
116116
- **Model Format:** pytorch
117117
- **Model Size (in billions):** 9
118118
- **Quantizations:** 8-bit, none
119-
- **Engines**: Transformers
119+
- **Engines**: vLLM, Transformers
120120
- **Model ID:** OpenGVLab/InternVL3-9B
121121
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-9B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-9B>`__
122122

@@ -132,7 +132,7 @@ Model Spec 8 (awq, 9 Billion)
132132
- **Model Format:** awq
133133
- **Model Size (in billions):** 9
134134
- **Quantizations:** Int4
135-
- **Engines**: Transformers
135+
- **Engines**: vLLM, Transformers
136136
- **Model ID:** OpenGVLab/InternVL3-9B-AWQ
137137
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-9B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-9B-AWQ>`__
138138

@@ -148,7 +148,7 @@ Model Spec 9 (pytorch, 14 Billion)
148148
- **Model Format:** pytorch
149149
- **Model Size (in billions):** 14
150150
- **Quantizations:** 8-bit, none
151-
- **Engines**: Transformers
151+
- **Engines**: vLLM, Transformers
152152
- **Model ID:** OpenGVLab/InternVL3-14B
153153
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-14B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-14B>`__
154154

@@ -164,7 +164,7 @@ Model Spec 10 (awq, 14 Billion)
164164
- **Model Format:** awq
165165
- **Model Size (in billions):** 14
166166
- **Quantizations:** Int4
167-
- **Engines**: Transformers
167+
- **Engines**: vLLM, Transformers
168168
- **Model ID:** OpenGVLab/InternVL3-14B-AWQ
169169
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-14B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-14B-AWQ>`__
170170

@@ -180,7 +180,7 @@ Model Spec 11 (pytorch, 38 Billion)
180180
- **Model Format:** pytorch
181181
- **Model Size (in billions):** 38
182182
- **Quantizations:** 8-bit, none
183-
- **Engines**: Transformers
183+
- **Engines**: vLLM, Transformers
184184
- **Model ID:** OpenGVLab/InternVL3-38B
185185
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-38B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-38B>`__
186186

@@ -196,7 +196,7 @@ Model Spec 12 (awq, 38 Billion)
196196
- **Model Format:** awq
197197
- **Model Size (in billions):** 38
198198
- **Quantizations:** Int4
199-
- **Engines**: Transformers
199+
- **Engines**: vLLM, Transformers
200200
- **Model ID:** OpenGVLab/InternVL3-38B-AWQ
201201
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-38B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-38B-AWQ>`__
202202

@@ -212,7 +212,7 @@ Model Spec 13 (pytorch, 78 Billion)
212212
- **Model Format:** pytorch
213213
- **Model Size (in billions):** 78
214214
- **Quantizations:** 8-bit, none
215-
- **Engines**: Transformers
215+
- **Engines**: vLLM, Transformers
216216
- **Model ID:** OpenGVLab/InternVL3-78B
217217
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-78B>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-78B>`__
218218

@@ -228,7 +228,7 @@ Model Spec 14 (awq, 78 Billion)
228228
- **Model Format:** awq
229229
- **Model Size (in billions):** 78
230230
- **Quantizations:** Int4
231-
- **Engines**: Transformers
231+
- **Engines**: vLLM, Transformers
232232
- **Model ID:** OpenGVLab/InternVL3-78B-AWQ
233233
- **Model Hubs**: `Hugging Face <https://huggingface.co/OpenGVLab/InternVL3-78B-AWQ>`__, `ModelScope <https://modelscope.cn/models/OpenGVLab/InternVL3-78B-AWQ>`__
234234

‎doc/source/models/builtin/llm/seallms-v3.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Model Spec 1 (pytorch, 1_5 Billion)
2020
- **Model Format:** pytorch
2121
- **Model Size (in billions):** 1_5
2222
- **Quantizations:** none
23-
- **Engines**: Transformers
23+
- **Engines**: vLLM, Transformers
2424
- **Model ID:** SeaLLMs/SeaLLMs-v3-1.5B-Chat
2525
- **Model Hubs**: `Hugging Face <https://huggingface.co/SeaLLMs/SeaLLMs-v3-1.5B-Chat>`__, `ModelScope <https://modelscope.cn/models/SeaLLMs/SeaLLMs-v3-1.5B-Chat>`__
2626

@@ -36,7 +36,7 @@ Model Spec 2 (pytorch, 7 Billion)
3636
- **Model Format:** pytorch
3737
- **Model Size (in billions):** 7
3838
- **Quantizations:** none
39-
- **Engines**: Transformers
39+
- **Engines**: vLLM, Transformers
4040
- **Model ID:** SeaLLMs/SeaLLMs-v3-7B-Chat
4141
- **Model Hubs**: `Hugging Face <https://huggingface.co/SeaLLMs/SeaLLMs-v3-7B-Chat>`__, `ModelScope <https://modelscope.cn/models/SeaLLMs/SeaLLMs-v3-7B-Chat>`__
4242

‎doc/source/models/builtin/llm/skywork-or1-preview.rst

+3-3
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Model Spec 1 (pytorch, 32 Billion)
2020
- **Model Format:** pytorch
2121
- **Model Size (in billions):** 32
2222
- **Quantizations:** none
23-
- **Engines**: Transformers
23+
- **Engines**: vLLM, Transformers
2424
- **Model ID:** Skywork/Skywork-OR1-32B-Preview
2525
- **Model Hubs**: `Hugging Face <https://huggingface.co/Skywork/Skywork-OR1-32B-Preview>`__, `ModelScope <https://modelscope.cn/models/Skywork/Skywork-OR1-32B-Preview>`__
2626

@@ -36,7 +36,7 @@ Model Spec 2 (gptq, 32 Billion)
3636
- **Model Format:** gptq
3737
- **Model Size (in billions):** 32
3838
- **Quantizations:** Int4, int8
39-
- **Engines**: Transformers
39+
- **Engines**: vLLM, Transformers
4040
- **Model ID:** JunHowie/Skywork-OR1-32B-Preview-GPTQ-{quantization}
4141
- **Model Hubs**: `Hugging Face <https://huggingface.co/JunHowie/Skywork-OR1-32B-Preview-GPTQ-{quantization}>`__, `ModelScope <https://modelscope.cn/models/JunHowie/Skywork-OR1-32B-Preview-GPTQ-{quantization}>`__
4242

@@ -52,7 +52,7 @@ Model Spec 3 (pytorch, 7 Billion)
5252
- **Model Format:** pytorch
5353
- **Model Size (in billions):** 7
5454
- **Quantizations:** none
55-
- **Engines**: Transformers
55+
- **Engines**: vLLM, Transformers
5656
- **Model ID:** Skywork/Skywork-OR1-7B-Preview
5757
- **Model Hubs**: `Hugging Face <https://huggingface.co/Skywork/Skywork-OR1-7B-Preview>`__, `ModelScope <https://modelscope.cn/models/Skywork/Skywork-OR1-7B-Preview>`__
5858

‎doc/source/models/virtualenv.rst

+11
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,15 @@ Solution
2121

2222
To address this issue, we have introduced the **Model Virtual Environment** feature.
2323

24+
Install requirements for this functionality via
25+
26+
.. code-block:: bash
27+
28+
# all
29+
pip install 'xinference[all]'
30+
# or virtualenv
31+
pip install 'xinference[virtualenv]'
32+
2433
Enable by setting environment variable ``XINFERENCE_ENABLE_VIRTUAL_ENV=1``.
2534

2635
Example usage:
@@ -37,6 +46,8 @@ Example usage:
3746

3847
This feature requires internet access or a self-hosted PyPI mirror.
3948

49+
Xinference will by default inherit the config for current pip.
50+
4051
.. note::
4152

4253
The model virtual environment feature is disabled by default (i.e., XINFERENCE_ENABLE_VIRTUAL_ENV is set to 0).

‎doc/source/user_guide/backends.rst

+2
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,8 @@ Currently, supported model includes:
7878
- ``QwQ-32B-Preview``, ``QwQ-32B``
7979
- ``marco-o1``
8080
- ``fin-r1``
81+
- ``seallms-v3``
82+
- ``skywork-or1-preview``
8183
- ``gemma-it``, ``gemma-2-it``, ``gemma-3-1b-it``
8284
- ``orion-chat``, ``orion-chat-rag``
8385
- ``c4ai-command-r-v01``

‎setup.cfg

+2
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,8 @@ doc =
274274
timm
275275
benchmark =
276276
psutil
277+
virtualenv =
278+
uv
277279

278280
[options.entry_points]
279281
console_scripts =

‎xinference/model/llm/llm_family.json

+3-1
Original file line numberDiff line numberDiff line change
@@ -7946,7 +7946,9 @@
79467946
"virtualenv": {
79477947
"packages": [
79487948
"git+https://github.com/huggingface/[email protected]",
7949-
"numpy==1.26.4"
7949+
"numpy==1.26.4",
7950+
"qwen_omni_utils",
7951+
"soundfile"
79507952
]
79517953
}
79527954
},

‎xinference/model/llm/llm_family_modelscope.json

+3-1
Original file line numberDiff line numberDiff line change
@@ -5704,7 +5704,9 @@
57045704
"virtualenv": {
57055705
"packages": [
57065706
"git+https://github.com/huggingface/[email protected]",
5707-
"numpy==1.26.4"
5707+
"numpy==1.26.4",
5708+
"qwen_omni_utils",
5709+
"soundfile"
57085710
]
57095711
}
57105712
},

0 commit comments

Comments
 (0)