You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
However, we do recommend you to handle them manually, because the download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).
Copy file name to clipboardExpand all lines: docker/llm/inference-cpp/README.md
+9-3
Original file line number
Diff line number
Diff line change
@@ -13,12 +13,18 @@
13
13
#### Setting Docker on windows
14
14
Need to enable `--net=host`,follow [this guide](https://docs.docker.com/network/drivers/host/#docker-desktop) so that you can easily access the service running on the docker. The [v6.1x kernel version wsl](https://learn.microsoft.com/en-us/community/content/wsl-user-msft-kernel-v6#1---building-the-microsoft-linux-kernel-v61x) is recommended to use.Otherwise, you may encounter the blocking issue before loading the model to GPU.
15
15
16
-
### Pull the latest image
16
+
### Build the Image
17
+
To build the `ipex-llm-inference-cpp-xpu` Docker image, use the following command:
To map the `xpu` into the container, you need to specify `--device=/dev/dri` when booting the container. Select the device you are running(device type:(Max, Flex, Arc, iGPU)). And change the `/path/to/models` to mount the models. `bench_model` is used to benchmark quickly. If want to benchmark, make sure it on the `/path/to/models`.
Copy file name to clipboardExpand all lines: docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md
+9-3
Original file line number
Diff line number
Diff line change
@@ -16,10 +16,16 @@
16
16
17
17
Need to enable `--net=host`,follow [this guide](https://docs.docker.com/network/drivers/host/#docker-desktop) so that you can easily access the service running on the docker. The [v6.1x kernel version wsl](https://learn.microsoft.com/en-us/community/content/wsl-user-msft-kernel-v6#1---building-the-microsoft-linux-kernel-v61x) is recommended to use.Otherwise, you may encounter the blocking issue before loading the model to GPU.
18
18
19
-
### Pull the latest image
19
+
### Build the Image
20
+
To build the `ipex-llm-inference-cpp-xpu` Docker image, use the following command:
Start ipex-llm-xpu Docker Container. Choose one of the following commands to start the container:
@@ -180,4 +185,4 @@ What is AI? [/INST]
180
185
<</SYS>>
181
186
182
187
What is AI? [/INST] Artificial intelligence (AI) is the broader field of research and development aimed at creating machines that can perform tasks that typically require human intelligence,
Copy file name to clipboardExpand all lines: docs/mddocs/DockerGuides/vllm_cpu_docker_quickstart.md
+8-4
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,17 @@ This guide demonstrates how to run `vLLM` serving with `ipex-llm` on Intel CPU v
6
6
7
7
Follow the instructions in this [guide](https://www.docker.com/get-started/) to install Docker on Linux.
8
8
9
-
## Pull the latest image
10
9
11
-
*Note: For running vLLM serving on Intel CPUs, you can currently use either the `intelanalytics/ipex-llm-serving-cpu:latest` or `intelanalytics/ipex-llm-serving-vllm-cpu:latest` Docker image.*
10
+
## Build the Image
11
+
To build the `ipex-llm-serving-cpu` Docker image, use the following command:
Copy file name to clipboardExpand all lines: docs/mddocs/DockerGuides/vllm_docker_quickstart.md
+8-5
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,16 @@ This guide demonstrates how to run `vLLM` serving with `IPEX-LLM` on Intel GPUs
6
6
7
7
Follow the instructions in this [guide](./docker_windows_gpu.md#linux) to install Docker on Linux.
8
8
9
-
## Pull the latest image
10
-
11
-
*Note: For running vLLM serving on Intel GPUs, you can currently use either the `intelanalytics/ipex-llm-serving-xpu:latest` or `intelanalytics/ipex-llm-serving-vllm-xpu:latest` Docker image.*
9
+
## Build the Image
10
+
To build the `ipex-llm-serving-xpu` Docker image, use the following command:
0 commit comments