Skip to content

Commit 1d7f4a8

Browse files
authored
Update documentation to build Docker image from Dockerfile instead of pulling from registry (#13057)
* Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update docker_cpp_xpu_quickstart.md * Update vllm_cpu_docker_quickstart.md * Update docker_cpp_xpu_quickstart.md * Update vllm_docker_quickstart.md * Update fastchat_docker_quickstart.md * Update docker_pytorch_inference_gpu.md
1 parent cd0d485 commit 1d7f4a8

File tree

9 files changed

+62
-37
lines changed

9 files changed

+62
-37
lines changed

docker/llm/finetune/xpu/README.md

+4-10
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,7 @@ With this docker image, we can use all [ipex-llm finetune examples on Intel GPU]
1616

1717
## 1. Prepare Docker Image
1818

19-
You can download directly from Dockerhub like:
20-
21-
```bash
22-
docker pull intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
23-
```
24-
25-
Or build the image from source:
19+
TO build the image from source:
2620

2721
```bash
2822
export HTTP_PROXY=your_http_proxy
@@ -31,7 +25,7 @@ export HTTPS_PROXY=your_https_proxy
3125
docker build \
3226
--build-arg http_proxy=${HTTP_PROXY} \
3327
--build-arg https_proxy=${HTTPS_PROXY} \
34-
-t intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT \
28+
-t intelanalytics/ipex-llm-finetune-xpu:latest \
3529
-f ./Dockerfile .
3630
```
3731

@@ -55,7 +49,7 @@ docker run -itd \
5549
-v $BASE_MODE_PATH:/model \
5650
-v $DATA_PATH:/data/alpaca-cleaned \
5751
--shm-size="16g" \
58-
intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
52+
intelanalytics/ipex-llm-finetune-xpu:latest
5953
```
6054

6155
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
@@ -72,7 +66,7 @@ docker run -itd \
7266
-e http_proxy=${HTTP_PROXY} \
7367
-e https_proxy=${HTTPS_PROXY} \
7468
--shm-size="16g" \
75-
intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
69+
intelanalytics/ipex-llm-finetune-xpu:latest
7670
```
7771

7872
However, we do recommend you to handle them manually, because the download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).

docker/llm/inference-cpp/README.md

+9-3
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,18 @@
1313
#### Setting Docker on windows
1414
Need to enable `--net=host`,follow [this guide](https://docs.docker.com/network/drivers/host/#docker-desktop) so that you can easily access the service running on the docker. The [v6.1x kernel version wsl]( https://learn.microsoft.com/en-us/community/content/wsl-user-msft-kernel-v6#1---building-the-microsoft-linux-kernel-v61x) is recommended to use.Otherwise, you may encounter the blocking issue before loading the model to GPU.
1515

16-
### Pull the latest image
16+
### Build the Image
17+
To build the `ipex-llm-inference-cpp-xpu` Docker image, use the following command:
18+
1719
```bash
18-
# This image will be updated every day
19-
docker pull intelanalytics/ipex-llm-inference-cpp-xpu:latest
20+
docker build \
21+
--build-arg http_proxy=.. \
22+
--build-arg https_proxy=.. \
23+
--build-arg no_proxy=.. \
24+
--rm --no-cache -t intelanalytics/ipex-llm-inference-cpp-xpu:latest .
2025
```
2126

27+
2228
### Start Docker Container
2329

2430
To map the `xpu` into the container, you need to specify `--device=/dev/dri` when booting the container. Select the device you are running(device type:(Max, Flex, Arc, iGPU)). And change the `/path/to/models` to mount the models. `bench_model` is used to benchmark quickly. If want to benchmark, make sure it on the `/path/to/models`.

docker/llm/serving/cpu/docker/README.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ docker build \
1414
--build-arg http_proxy=.. \
1515
--build-arg https_proxy=.. \
1616
--build-arg no_proxy=.. \
17-
--rm --no-cache -t intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT .
17+
--rm --no-cache -t intelanalytics/ipex-llm-serving-cpu:latest .
1818
```
1919

2020
---
@@ -40,7 +40,7 @@ This ensures the container has access to the necessary models.
4040
Use the following command to start the container:
4141

4242
```bash
43-
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
43+
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:latest
4444

4545
sudo docker run -itd \
4646
--net=host \ # Use host networking for performance
@@ -151,4 +151,4 @@ Then, follow these steps:
151151
--load-in-low-bit sym_int4
152152
```
153153

154-
---
154+
---

docker/llm/serving/xpu/docker/README.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ docker build \
1313
--build-arg http_proxy=.. \
1414
--build-arg https_proxy=.. \
1515
--build-arg no_proxy=.. \
16-
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT .
16+
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:latest .
1717
```
1818

1919
---
@@ -26,11 +26,12 @@ To map the `XPU` into the container, you need to specify `--device=/dev/dri` whe
2626

2727
```bash
2828
#/bin/bash
29-
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
29+
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:latest
3030

3131
sudo docker run -itd \
3232
--net=host \
3333
--device=/dev/dri \
34+
--privileged \
3435
--memory="32G" \
3536
--name=CONTAINER_NAME \
3637
--shm-size="16g" \
@@ -72,7 +73,7 @@ By default, the container is configured to automatically start the service when
7273

7374
```bash
7475
#/bin/bash
75-
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
76+
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:latest
7677

7778
sudo docker run -itd \
7879
--net=host \
@@ -112,11 +113,12 @@ If you prefer to manually start the service or need to troubleshoot, you can ove
112113

113114
```bash
114115
#/bin/bash
115-
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
116+
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:latest
116117

117118
sudo docker run -itd \
118119
--net=host \
119120
--device=/dev/dri \
121+
--privileged \
120122
--memory="32G" \
121123
--name=CONTAINER_NAME \
122124
--shm-size="16g" \

docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md

+9-3
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,16 @@
1616

1717
Need to enable `--net=host`,follow [this guide](https://docs.docker.com/network/drivers/host/#docker-desktop) so that you can easily access the service running on the docker. The [v6.1x kernel version wsl]( https://learn.microsoft.com/en-us/community/content/wsl-user-msft-kernel-v6#1---building-the-microsoft-linux-kernel-v61x) is recommended to use.Otherwise, you may encounter the blocking issue before loading the model to GPU.
1818

19-
### Pull the latest image
19+
### Build the Image
20+
To build the `ipex-llm-inference-cpp-xpu` Docker image, use the following command:
21+
2022
```bash
21-
# This image will be updated every day
22-
docker pull intelanalytics/ipex-llm-inference-cpp-xpu:latest
23+
cd docker/llm/inference-cpp
24+
docker build \
25+
--build-arg http_proxy=.. \
26+
--build-arg https_proxy=.. \
27+
--build-arg no_proxy=.. \
28+
--rm --no-cache -t intelanalytics/ipex-llm-inference-cpp-xpu:latest .
2329
```
2430

2531
### Start Docker Container

docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md

+7-2
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,12 @@ Follow the [Docker installation Guide](./docker_windows_gpu.md#install-docker) t
1313

1414
Prepare ipex-llm-serving-xpu Docker Image:
1515
```bash
16-
docker pull intelanalytics/ipex-llm-serving-xpu:latest
16+
cd docker/llm/serving/xpu/docker
17+
docker build \
18+
--build-arg http_proxy=.. \
19+
--build-arg https_proxy=.. \
20+
--build-arg no_proxy=.. \
21+
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:latest .
1722
```
1823

1924
Start ipex-llm-xpu Docker Container. Choose one of the following commands to start the container:
@@ -180,4 +185,4 @@ What is AI? [/INST]
180185
<</SYS>>
181186
182187
What is AI? [/INST] Artificial intelligence (AI) is the broader field of research and development aimed at creating machines that can perform tasks that typically require human intelligence,
183-
```
188+
```

docs/mddocs/DockerGuides/fastchat_docker_quickstart.md

+8-3
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,16 @@ This guide demonstrates how to run `FastChat` serving with `IPEX-LLM` on Intel G
66

77
Follow the instructions in this [guide](./docker_windows_gpu.md#linux) to install Docker on Linux.
88

9-
## Pull the latest image
9+
## Build the Image
10+
To build the `ipex-llm-serving-xpu` Docker image, use the following command:
1011

1112
```bash
12-
# This image will be updated every day
13-
docker pull intelanalytics/ipex-llm-serving-xpu:latest
13+
cd docker/llm/serving/xpu/docker
14+
docker build \
15+
--build-arg http_proxy=.. \
16+
--build-arg https_proxy=.. \
17+
--build-arg no_proxy=.. \
18+
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:latest .
1419
```
1520

1621
## Start Docker Container

docs/mddocs/DockerGuides/vllm_cpu_docker_quickstart.md

+8-4
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,17 @@ This guide demonstrates how to run `vLLM` serving with `ipex-llm` on Intel CPU v
66

77
Follow the instructions in this [guide](https://www.docker.com/get-started/) to install Docker on Linux.
88

9-
## Pull the latest image
109

11-
*Note: For running vLLM serving on Intel CPUs, you can currently use either the `intelanalytics/ipex-llm-serving-cpu:latest` or `intelanalytics/ipex-llm-serving-vllm-cpu:latest` Docker image.*
10+
## Build the Image
11+
To build the `ipex-llm-serving-cpu` Docker image, use the following command:
1212

1313
```bash
14-
# This image will be updated every day
15-
docker pull intelanalytics/ipex-llm-serving-cpu:latest
14+
cd docker/llm/serving/cpu/docker
15+
docker build \
16+
--build-arg http_proxy=.. \
17+
--build-arg https_proxy=.. \
18+
--build-arg no_proxy=.. \
19+
--rm --no-cache -t intelanalytics/ipex-llm-serving-cpu:latest .
1620
```
1721

1822
## Start Docker Container

docs/mddocs/DockerGuides/vllm_docker_quickstart.md

+8-5
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,16 @@ This guide demonstrates how to run `vLLM` serving with `IPEX-LLM` on Intel GPUs
66

77
Follow the instructions in this [guide](./docker_windows_gpu.md#linux) to install Docker on Linux.
88

9-
## Pull the latest image
10-
11-
*Note: For running vLLM serving on Intel GPUs, you can currently use either the `intelanalytics/ipex-llm-serving-xpu:latest` or `intelanalytics/ipex-llm-serving-vllm-xpu:latest` Docker image.*
9+
## Build the Image
10+
To build the `ipex-llm-serving-xpu` Docker image, use the following command:
1211

1312
```bash
14-
# This image will be updated every day
15-
docker pull intelanalytics/ipex-llm-serving-xpu:latest
13+
cd docker/llm/serving/xpu/docker
14+
docker build \
15+
--build-arg http_proxy=.. \
16+
--build-arg https_proxy=.. \
17+
--build-arg no_proxy=.. \
18+
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:latest .
1619
```
1720

1821
## Start Docker Container

0 commit comments

Comments
 (0)