DOC: add doc about virtual env & update models in README (#3287)

Co-authored-by: codingl2k1 <[email protected]>

Author: qinxuye

README.md

@@ -47,14 +47,14 @@ potential of cutting-edge AI models.
 - Support SGLang backend: [#1161](https://github.com/xorbitsai/inference/pull/1161)
 - Support LoRA for LLM and image models: [#1080](https://github.com/xorbitsai/inference/pull/1080)
 ### New Models
-- Built-in support for [Gemma-3-it](https://blog.google/technology/developers/gemma-3/): [#3077](https://github.com/xorbitsai/inference/pull/3077)
-- Built-in support for [QwQ-32B](https://qwenlm.github.io/blog/qwq-32b/): [#3005](https://github.com/xorbitsai/inference/pull/3005)
-- Built-in support for [DeepSeek V3 and R1](https://github.com/deepseek-ai/DeepSeek-R1): [#2864](https://github.com/xorbitsai/inference/pull/2864)
-- Built-in support for [InternVL2.5](https://internvl.github.io/blog/2024-12-05-InternVL-2.5/): [#2776](https://github.com/xorbitsai/inference/pull/2776)
-- Built-in support for [DeepSeek-R1-Distill-Llama](https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#deepseek-r1-distill-models): [#2811](https://github.com/xorbitsai/inference/pull/2811)
-- Built-in support for [DeepSeek-R1-Distill-Qwen](https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#deepseek-r1-distill-models): [#2781](https://github.com/xorbitsai/inference/pull/2781)
-- Built-in support for [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M): [#2790](https://github.com/xorbitsai/inference/pull/2790)
-- Built-in support for [qwen2.5-vl](https://github.com/QwenLM/Qwen2.5-VL): [#2788](https://github.com/xorbitsai/inference/pull/2788)
+- Built-in support for [Qwen2.5-Omni](https://github.com/QwenLM/Qwen2.5-Omni): [#3279](https://github.com/xorbitsai/inference/pull/3279)
+- Built-in support for [Skywork-OR1](https://github.com/SkyworkAI/Skywork-OR1): [#3274](https://github.com/xorbitsai/inference/pull/3274)
+- Built-in support for [GLM-4-0414](https://github.com/THUDM/GLM-4): [#3251](https://github.com/xorbitsai/inference/pull/3251)
+- Built-in support for [SeaLLMs-v3](https://github.com/DAMO-NLP-SG/DAMO-SeaLLMs): [#3248](https://github.com/xorbitsai/inference/pull/3248)
+- Built-in support for [paraformer-zh](https://huggingface.co/funasr/paraformer-zh): [#3236](https://github.com/xorbitsai/inference/pull/3236)
+- Built-in support for [InternVL3](https://internvl.github.io/blog/2025-04-11-InternVL-3.0/): [#3235](https://github.com/xorbitsai/inference/pull/3235)
+- Built-in support for [MegaTTS3](https://github.com/bytedance/MegaTTS3): [#3224](https://github.com/xorbitsai/inference/pull/3224)
+- Built-in support for [Deepseek-VL2](https://github.com/deepseek-ai/DeepSeek-VL2): [#3179](https://github.com/xorbitsai/inference/pull/3179)
 ### Integrations
 - [Dify](https://docs.dify.ai/advanced/model-configuration/xinference): an LLMOps platform that enables developers (and even non-developers) to quickly build useful applications based on large language models, ensuring they are visual, operable, and improvable.
 - [FastGPT](https://github.com/labring/FastGPT): a knowledge-based platform built on the LLM, offers out-of-the-box data processing and model invocation capabilities, allows for workflow orchestration through Flow visualization.

README_zh_CN.md

@@ -43,14 +43,14 @@ Xorbits Inference（Xinference）是一个性能强大且功能全面的分布
 - 支持 SGLang 后端: [#1161](https://github.com/xorbitsai/inference/pull/1161)
 - 支持LLM和图像模型的LoRA: [#1080](https://github.com/xorbitsai/inference/pull/1080)
 ### 新模型
-- 内置 [Gemma-3-it](https://blog.google/technology/developers/gemma-3/): [#3077](https://github.com/xorbitsai/inference/pull/3077)
-- 内置 [QwQ-32B](https://qwenlm.github.io/zh/blog/qwq-32b/): [#3005](https://github.com/xorbitsai/inference/pull/3005)
-- 内置 [DeepSeek V3 and R1](https://github.com/deepseek-ai/DeepSeek-R1): [#2864](https://github.com/xorbitsai/inference/pull/2864)
-- 内置 [InternVL2.5](https://internvl.github.io/blog/2024-12-05-InternVL-2.5/): [#2776](https://github.com/xorbitsai/inference/pull/2776)
-- 内置 [DeepSeek-R1-Distill-Llama](https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#deepseek-r1-distill-models): [#2811](https://github.com/xorbitsai/inference/pull/2811)
-- 内置 [DeepSeek-R1-Distill-Qwen](https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-file#deepseek-r1-distill-models): [#2781](https://github.com/xorbitsai/inference/pull/2781)
-- 内置 [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M): [#2790](https://github.com/xorbitsai/inference/pull/2790)
-- 内置 [qwen2.5-vl](https://github.com/QwenLM/Qwen2.5-VL): [#2788](https://github.com/xorbitsai/inference/pull/2788)
+- 内置 [Qwen2.5-Omni](https://github.com/QwenLM/Qwen2.5-Omni): [#3279](https://github.com/xorbitsai/inference/pull/3279)
+- 内置 [Skywork-OR1](https://github.com/SkyworkAI/Skywork-OR1): [#3274](https://github.com/xorbitsai/inference/pull/3274)
+- 内置 [GLM-4-0414](https://github.com/THUDM/GLM-4): [#3251](https://github.com/xorbitsai/inference/pull/3251)
+- 内置 [SeaLLMs-v3](https://github.com/DAMO-NLP-SG/DAMO-SeaLLMs): [#3248](https://github.com/xorbitsai/inference/pull/3248)
+- 内置 [paraformer-zh](https://huggingface.co/funasr/paraformer-zh): [#3236](https://github.com/xorbitsai/inference/pull/3236)
+- 内置 [InternVL3](https://internvl.github.io/blog/2025-04-11-InternVL-3.0/): [#3235](https://github.com/xorbitsai/inference/pull/3235)
+- 内置 [MegaTTS3](https://github.com/bytedance/MegaTTS3): [#3224](https://github.com/xorbitsai/inference/pull/3224)
+- 内置 [Deepseek-VL2](https://github.com/deepseek-ai/DeepSeek-VL2): [#3179](https://github.com/xorbitsai/inference/pull/3179)
 ### 集成
 - [FastGPT](https://doc.fastai.site/docs/development/custom-models/xinference/)：一个基于 LLM 大模型的开源 AI 知识库构建平台。提供了开箱即用的数据处理、模型调用、RAG 检索、可视化 AI 工作流编排等能力，帮助您轻松实现复杂的问答场景。
 - [Dify](https://docs.dify.ai/advanced/model-configuration/xinference): 一个涵盖了大型语言模型开发、部署、维护和优化的 LLMOps 平台。

doc/source/getting_started/environments.rst

@@ -14,6 +14,8 @@ XINFERENCE_MODEL_SRC
 Modelhub used for downloading models. Default is "huggingface", or you
 can set "modelscope" as downloading source.
 
+.. _environments_xinference_home:
+
 XINFERENCE_HOME
 ~~~~~~~~~~~~~~~~
 By default, Xinference uses ``<HOME>/.xinference`` as home path to store

doc/source/getting_started/installation.rst

@@ -89,17 +89,12 @@ and will be the sole backend for llama.cpp in the future.
 
 .. note::
 
-    ``llama-cpp-python`` is the default option for llama.cpp backend.
-    To enable xllamacpp, add environment variable ``USE_XLLAMACPP=1``.
-
-    e.g. Starting local Xinference via
-
-    ``USE_XLLAMACPP=1 xinference-local``
+    ``xllamacpp`` is the default option for llama.cpp backend since v1.5.0.
+    To enable ``llama-cpp-python``, add environment variable ``USE_XLLAMACPP=0``.
 
 .. warning::
 
-    For upcoming Xinference v1.5.0,
-    ``xllamacpp`` will become default option for llama.cpp, and ``llama-cpp-python`` will be deprecated.
+    Since Xinference v1.5.0, ``llama-cpp-python`` will be deprecated.
     For Xinference v1.6.0, ``llama-cpp-python`` will be removed.
 
 Initial setup::
@@ -112,10 +107,14 @@ Installation instructions for ``xllamacpp``:
 
    pip install -U xllamacpp
 
-- Cuda::
+- CUDA::
 
    pip install xllamacpp --force-reinstall --index-url https://xorbitsai.github.io/xllamacpp/whl/cu124
 
+- HIP::
+
+   pip install xllamacpp --force-reinstall --index-url https://xorbitsai.github.io/xllamacpp/whl/rocm-6.0.2
+
 Hardware-Specific installations for ``llama-cpp-python``:
 
 - Apple Silicon::

doc/source/locale/zh_CN/LC_MESSAGES/getting_started/installation.po

@@ -7,7 +7,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2025-03-19 12:51+0800\n"
+"POT-Creation-Date: 2025-04-19 00:40+0200\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: zh_CN\n"
@@ -16,7 +16,7 @@ msgstr ""
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=utf-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Babel 2.14.0\n"
+"Generated-By: Babel 2.16.0\n"
 
 #: ../../source/getting_started/installation.rst:5
 msgid "Installation"
@@ -160,7 +160,7 @@ msgstr ""
 #: ../../source/getting_started/installation.rst:50
 msgid ""
 "``qwen2.5``, ``qwen2.5-coder``, ``qwen2.5-instruct``, ``qwen2.5-coder-"
-"instruct``"
+"instruct``, ``qwen2.5-instruct-1m``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:51
@@ -212,86 +212,89 @@ msgid "``marco-o1``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:63
-msgid "``gemma-it``, ``gemma-2-it``"
+msgid "``fin-r1``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:64
-msgid "``orion-chat``, ``orion-chat-rag``"
+msgid "``gemma-it``, ``gemma-2-it``, ``gemma-3-1b-it``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:65
-msgid "``c4ai-command-r-v01``"
+msgid "``orion-chat``, ``orion-chat-rag``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:66
-msgid "``minicpm3-4b``"
+msgid "``c4ai-command-r-v01``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:67
-msgid "``internlm3-instruct``"
+msgid "``minicpm3-4b``"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:68
+msgid "``internlm3-instruct``"
+msgstr ""
+
+#: ../../source/getting_started/installation.rst:69
 msgid "``moonlight-16b-a3b-instruct``"
 msgstr ""
 
-#: ../../source/getting_started/installation.rst:71
+#: ../../source/getting_started/installation.rst:72
 msgid "To install Xinference and vLLM::"
 msgstr "安装 xinference 和 vLLM："
 
-#: ../../source/getting_started/installation.rst:84
+#: ../../source/getting_started/installation.rst:85
 msgid "Llama.cpp Backend"
 msgstr "Llama.cpp 引擎"
 
-#: ../../source/getting_started/installation.rst:85
+#: ../../source/getting_started/installation.rst:86
 msgid ""
 "Xinference supports models in ``gguf`` format via ``xllamacpp`` or "
 "``llama-cpp-python``. `xllamacpp "
 "<https://github.com/xorbitsai/xllamacpp>`_ is developed by Xinference "
 "team, and will be the sole backend for llama.cpp in the future."
-msgstr "Xinference 通过 xllamacpp 或 llama-cpp-python 支持 gguf 格式的模型。"
-"`xllamacpp <https://github.com/xorbitsai/xllamacpp>`_ 由 Xinference 团队开发，"
-"并将在未来成为 llama.cpp 的唯一后端。"
+msgstr ""
+"Xinference 通过 xllamacpp 或 llama-cpp-python 支持 gguf 格式的模型。`"
+"xllamacpp <https://github.com/xorbitsai/xllamacpp>`_ 由 Xinference 团队"
+"开发，并将在未来成为 llama.cpp 的唯一后端。"
 
-#: ../../source/getting_started/installation.rst:91
+#: ../../source/getting_started/installation.rst:92
 msgid ""
-"``llama-cpp-python`` is the default option for llama.cpp backend. To "
-"enable xllamacpp, add environment variable ``USE_XLLAMACPP=1``."
-msgstr "``llama-cpp-python`` 是 llama.cpp 后端的默认选项。"
-"要启用 xllamacpp，请添加环境变量 USE_XLLAMACPP=1。"
-
-#: ../../source/getting_started/installation.rst:94
-msgid "e.g. Starting local Xinference via"
-msgstr "例如，通过以下方式启动本地 Xinference"
-
-#: ../../source/getting_started/installation.rst:96
-msgid "``USE_XLLAMACPP=1 xinference-local``"
+"``xllamacpp`` is the default option for llama.cpp backend since v1.5.0. "
+"To enable ``llama-cpp-python``, add environment variable "
+"``USE_XLLAMACPP=0``."
 msgstr ""
+"自 v1.5.0 起，``xllamacpp`` 成为 llama.cpp 后端的默认选项。如需启用 ``"
+"llama-cpp-python``，请设置环境变量 ``USE_XLLAMACPP=0``。"
 
-#: ../../source/getting_started/installation.rst:100
+#: ../../source/getting_started/installation.rst:97
 msgid ""
-"For upcoming Xinference v1.5.0, ``xllamacpp`` will become default option "
-"for llama.cpp, and ``llama-cpp-python`` will be deprecated. For "
+"Since Xinference v1.5.0, ``llama-cpp-python`` will be deprecated. For "
 "Xinference v1.6.0, ``llama-cpp-python`` will be removed."
-msgstr "在即将发布的 Xinference v1.5.0 中，``xllamacpp`` 将成为 llama.cpp 的默认选项，"
-"而 ``llama-cpp-python`` 将被弃用。在 Xinference v1.6.0 中，``llama-cpp-python`` 将被移除。"
+msgstr ""
+"自 Xinference v1.5.0 起，``llama-cpp-python`` 将被弃用；在 Xinference "
+"v1.6.0 中，该后端将被移除。"
 
-#: ../../source/getting_started/installation.rst:104
+#: ../../source/getting_started/installation.rst:100
 #: ../../source/getting_started/installation.rst:137
 #: ../../source/getting_started/installation.rst:150
 msgid "Initial setup::"
 msgstr "初始步骤："
 
-#: ../../source/getting_started/installation.rst:108
+#: ../../source/getting_started/installation.rst:104
 msgid "Installation instructions for ``xllamacpp``:"
 msgstr "``xllamacpp`` 的安装说明："
 
-#: ../../source/getting_started/installation.rst:110
+#: ../../source/getting_started/installation.rst:106
 msgid "CPU or Mac Metal::"
 msgstr "CPU 或 Mac Metal："
 
+#: ../../source/getting_started/installation.rst:110
+msgid "CUDA::"
+msgstr ""
+
 #: ../../source/getting_started/installation.rst:114
-msgid "Cuda::"
+msgid "HIP::"
 msgstr ""
 
 #: ../../source/getting_started/installation.rst:118
@@ -364,3 +367,20 @@ msgstr ""
 #~ "建议根据当前使用的硬件手动安装依赖，从而"
 #~ "获得最佳的加速效果。"
 
+#~ msgid ""
+#~ "``qwen2.5``, ``qwen2.5-coder``, ``qwen2.5-instruct``, "
+#~ "``qwen2.5-coder-instruct``"
+#~ msgstr ""
+
+#~ msgid "``gemma-it``, ``gemma-2-it``"
+#~ msgstr ""
+
+#~ msgid "e.g. Starting local Xinference via"
+#~ msgstr "例如，通过以下方式启动本地 Xinference"
+
+#~ msgid "``USE_XLLAMACPP=1 xinference-local``"
+#~ msgstr ""
+
+#~ msgid "Cuda::"
+#~ msgstr ""
+

doc/source/locale/zh_CN/LC_MESSAGES/models/index.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2025-01-26 11:51+0800\n"
+"POT-Creation-Date: 2025-04-19 00:37+0800\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: zh_CN\n"
@@ -140,70 +140,78 @@ msgid ""
 msgstr "当你不再需要当前正在运行的模型时，以下列方式释放其占用的资源："
 
 #: ../../source/models/index.rst:161
+msgid ""
+"For models that are no longer maintained and depend on outdated libraries"
+" (such as ``transformers``), we recommend enabling the :ref:`Model "
+"Virtual Environment <model_virtual_env>` feature to ensure they can run "
+"properly in a compatible environment."
+msgstr "对于不再维护且依赖旧版库（如 ``transformers`` ）的模型，建议启用 :ref:`模型虚拟空间 <model_virtual_env>` 功能，以确保它们能在兼容的环境中正常运行。"
+
+#: ../../source/models/index.rst:167
 msgid "Model Usage"
 msgstr "模型使用"
 
-#: ../../source/models/index.rst:166
+#: ../../source/models/index.rst:172
 msgid "Chat & Generate"
 msgstr "聊天 & 生成"
 
-#: ../../source/models/index.rst:170
+#: ../../source/models/index.rst:176
 msgid "Learn how to chat with LLMs in Xinference."
 msgstr "学习如何在 Xinference 中与 LLM聊天。"
 
-#: ../../source/models/index.rst:172
+#: ../../source/models/index.rst:178
 msgid "Tools"
 msgstr "工具"
 
-#: ../../source/models/index.rst:176
+#: ../../source/models/index.rst:182
 msgid "Learn how to connect LLM with external tools."
 msgstr "学习如何将 LLM 与外部工具连接起来。"
 
-#: ../../source/models/index.rst:181
+#: ../../source/models/index.rst:187
 msgid "Embeddings"
 msgstr "嵌入"
 
-#: ../../source/models/index.rst:185
+#: ../../source/models/index.rst:191
 msgid "Learn how to create text embeddings in Xinference."
 msgstr "学习如何在 Xinference 中创建文本嵌入。"
 
-#: ../../source/models/index.rst:187
+#: ../../source/models/index.rst:193
 msgid "Rerank"
 msgstr "重排序"
 
-#: ../../source/models/index.rst:191
+#: ../../source/models/index.rst:197
 msgid "Learn how to use rerank models in Xinference."
 msgstr "学习如何在 Xinference 中使用重排序模型。"
 
-#: ../../source/models/index.rst:196
+#: ../../source/models/index.rst:202
 msgid "Images"
 msgstr "图像"
 
-#: ../../source/models/index.rst:200
+#: ../../source/models/index.rst:206
 msgid "Learn how to generate images with Xinference."
 msgstr "学习如何使用Xinference生成图像。"
 
-#: ../../source/models/index.rst:202
+#: ../../source/models/index.rst:208
 msgid "Multimodal"
 msgstr "多模态"
 
-#: ../../source/models/index.rst:206
+#: ../../source/models/index.rst:212
 msgid "Learn how to process images and audio with LLMs."
 msgstr "学习如何使用 LLM 处理图像和音频。"
 
-#: ../../source/models/index.rst:211
+#: ../../source/models/index.rst:217
 msgid "Audio"
 msgstr "音频"
 
-#: ../../source/models/index.rst:215
+#: ../../source/models/index.rst:221
 msgid "Learn how to turn audio into text or text into audio with Xinference."
 msgstr "学习如何使用 Xinference 将音频转换为文本或将文本转换为音频。"
 
-#: ../../source/models/index.rst:217
+#: ../../source/models/index.rst:223
 msgid "Video"
 msgstr "视频"
 
-#: ../../source/models/index.rst:221
+#: ../../source/models/index.rst:227
 msgid "Learn how to generate video with Xinference."
 msgstr "学习如何使用Xinference生成视频。"