Skip to content

Commit 932373e

Browse files
mssonicbldmssonicbld
authored
Use pzstd to compress the Docker in SWI slim images (#21869)
<!-- Please make sure you've read and understood our contributing guidelines: https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md ** Make sure all your commits include a signature generated with `git commit -s` ** If this is a bug fix, make sure your description includes "fixes #xxxx", or "closes #xxxx" or "resolves #xxxx" Please provide the following information: --> #### Why I did it Same with #21824 Enabled the docker inram feature for slim image. It would extract the docker image to ram during the boot, so it would take extra times during boot. ##### Work item tracking - Microsoft ADO **(number only)**: 31323281 #### How I did it Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time. Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed. We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that. So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file. Plan to support zstd for all types of images in the future to unify the docker file image . #### How to verify it Boot swi slim image and normal image, boot mellanox bin image, all boot successfully. <!-- If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012. --> #### Which release branch to backport (provide reason below if selected) <!-- - Note we only backport fixes to a release branch, *not* features! - Please also provide a reason for the backporting below. - e.g. - [x] 202006 --> - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 - [ ] 202211 - [ ] 202305 #### Tested branch (Please provide the tested image version) <!-- - Please provide tested image version - e.g. - [x] 20201231.100 --> - [ ] <!-- image version 1 --> - [ ] <!-- image version 2 --> #### Description for the changelog <!-- Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> <!-- Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU. --> #### Link to config_db schema for YANG module changes <!-- Provide a link to config_db schema for the table for which YANG model is defined Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md --> #### A picture of a cute animal (not mandatory but encouraged)
1 parent f3ddb66 commit 932373e

File tree

5 files changed

+58
-4
lines changed

5 files changed

+58
-4
lines changed

‎build_debian.sh

+11-1
Original file line numberDiff line numberDiff line change
@@ -409,6 +409,12 @@ sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y in
409409
zstd \
410410
nvme-cli
411411

412+
sudo cp files/initramfs-tools/pzstd $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/pzstd
413+
sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/pzstd
414+
415+
sudo cp files/initramfs-tools/file $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/file
416+
sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/file
417+
412418
# Have systemd create the auditd log directory
413419
sudo mkdir -p ${FILESYSTEM_ROOT}/etc/systemd/system/auditd.service.d
414420
sudo tee ${FILESYSTEM_ROOT}/etc/systemd/system/auditd.service.d/log-directory.conf >/dev/null <<EOF
@@ -872,7 +878,11 @@ if [[ $MULTIARCH_QEMU_ENVIRON == y || $CROSS_BUILD_ENVIRON == y ]]; then
872878
fi
873879

874880
## Compress docker files
875-
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
881+
if [ "$BUILD_REDUCE_IMAGE_SIZE" = "y" ]; then
882+
pushd $FILESYSTEM_ROOT && sudo tar -I pzstd -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
883+
else
884+
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
885+
fi
876886

877887
## Compress together with /boot, /var/lib/docker and $PLATFORM_DIR as an installer payload zip file
878888
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf platform.tar.gz -C $PLATFORM_DIR . && sudo zip -n .gz $OLDPWD/$INSTALLER_PAYLOAD -r boot/ platform.tar.gz; popd

‎files/dsc/install_debian.j2

+4
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,11 @@ image_dir=image-$image_version
1515

1616
INSTALLER_PAYLOAD=fs.zip
1717
DOCKERFS_DIR=docker
18+
{% if BUILD_REDUCE_IMAGE_SIZE == "y" -%}
19+
FILESYSTEM_DOCKERFS=dockerfs.tar.zstd
20+
{%- else -%}
1821
FILESYSTEM_DOCKERFS=dockerfs.tar.gz
22+
{%- endif %}
1923
BL_CONF=boot.conf
2024

2125
DATA_PARTUUID=6ED62003-DD8D-44B8-9538-0A2B7C7E628F

‎files/initramfs-tools/file

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/sh
2+
set -e
3+
PREREQ=""
4+
prereqs() {
5+
echo "$PREREQ"
6+
}
7+
case "$1" in
8+
prereqs)
9+
prereqs
10+
exit 0
11+
;;
12+
esac
13+
. /usr/share/initramfs-tools/hook-functions
14+
# Include file binary
15+
copy_exec /usr/bin/file /usr/bin
16+
# Include magic database
17+
copy_exec /usr/lib/file/magic.mgc /etc
18+
exit 0

‎files/initramfs-tools/pzstd

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/bin/sh
2+
set -e
3+
PREREQ=""
4+
prereqs() {
5+
echo "$PREREQ"
6+
}
7+
case "$1" in
8+
prereqs)
9+
prereqs
10+
exit 0
11+
;;
12+
esac
13+
. /usr/share/initramfs-tools/hook-functions
14+
# Include pzstd binary
15+
copy_exec /usr/bin/pzstd /usr/bin
16+
exit 0

‎files/initramfs-tools/union-mount.j2

+9-3
Original file line numberDiff line numberDiff line change
@@ -130,9 +130,15 @@ extract_dockerfs()
130130
{
131131
echo "Extracting {{ FILESYSTEM_DOCKERFS }}"
132132
if [ -f "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}" ] && [ "$secureboot" = false ]; then
133-
# Extract dockerfs.tar.gz into /var/lib/docker unless the system booted with secureboot
134-
# In secureboot dockerfs.tar.gz cannot be trusted as it does not have a signature
135-
tar xz --numeric-owner -f ${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }} -C ${rootmnt}/var/lib/docker
133+
# Check if the file is zstd compressed
134+
file_type=$(file -b --mime-type "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}")
135+
if [ "$file_type" = "application/zstd" ]; then
136+
echo "Detected zstd compression, extracting with pzstd..."
137+
pzstd -d -q ${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }} -c | tar x --numeric-owner -C ${rootmnt}/var/lib/docker
138+
else
139+
echo "Using default extraction method (gzip assumed)..."
140+
tar xz --numeric-owner -f "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}" -C "${rootmnt}/var/lib/docker"
141+
fi
136142
elif [ "$bootloader" = "aboot" ] && unzip -l "$swi_path" | grep -q {{ FILESYSTEM_DOCKERFS }}; then
137143
# Aboot swi images also support extracting dockerfs.tar.gz directly from them
138144
unzip -qp "$swi_path" {{ FILESYSTEM_DOCKERFS }} | tar xz --numeric-owner -C ${rootmnt}/var/lib/docker

0 commit comments

Comments
 (0)