Install azure keyvault python package for k8s master image #17806

lixiaoyuner · 2024-01-17T08:24:55Z

Why I did it

Currently we will use AAD to do authentication with python language inside k8s master, we need to pre-install azure key-vault python packages.
Found some issues to fix when I tried to test k8s master image build.

Need to install gnupg package before installing GPG key for bookworm
The netcat package's name has changed to netcat-openbsd in bookworm
Upgrade the cri-dockerd version to 0.3.10 for bookworm
Need to build a VHDX disk for convenience of image usage
Change the k8s master change flag name to make it easy to understand
Remove the -x for the skipvstest script due to it will add an extra quota when the variables are referred.

Work item tracking

Microsoft ADO (number only): 26435886

How I did it

pip3 install azure-keyvault-secrets
apt-get -y install gnupg before install GPG key
apt-get -y install netcat-openbsd
upgrade the cri-dockerd version for bookworm

How to verify it

pip3 list to check if azure-keyvault-secrets is installed inside image
dpkg -l to check if gnupg and netcat-openbsd is installed inside image
systemctl status cri-dockerd.service to check if it's running well

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Yun Li <[email protected]>

xumia · 2024-01-25T04:19:29Z

.azure-pipelines/azure-pipelines-build.yml

-              make $BUILD_OPTIONS $(K8S_OPTIONS) target/sonic-vs.img.gz
-              mv target/sonic-vs.img.gz target/sonic-vs-k8s.img.gz
+              make $BUILD_OPTIONS INCLUDE_KUBERNETES_MASTER=y target/sonic-vs.img.gz
+              gzip -kd target/sonic-vs.img.gz


Looks like the keep option "-k" can be removed.

Looks like the keep option "-k" can be removed.

Yes, you are correct. Thanks for this comment, fixed.

xumia

LGTM

losha228

LGTM

qiluo-msft · 2024-01-30T22:15:48Z

build_debian.sh

@@ -271,6 +271,8 @@ sudo LANG=C chroot $FILESYSTEM_ROOT apt-get -y install docker-ce=${DOCKER_VERSIO

 install_kubernetes () {
    local ver="$1"
+    sudo LANG=C chroot $FILESYSTEM_ROOT apt-get update


apt-get update

No need to update again. #Closed

apt-get update

No need to update again.

Removed

qiluo-msft · 2024-01-30T22:15:57Z

sudo LANG=C chroot $FILESYSTEM_ROOT apt-get update

No need to update again. #Closed

Refers to: build_debian.sh:281 in e65bae4. [](commit_id = e65bae4, deletion_comment = False)

qiluo-msft · 2024-01-30T22:16:06Z

build_debian.sh

@@ -299,10 +301,9 @@ then
    install_kubernetes ${MASTER_KUBERNETES_VERSION}

    sudo LANG=C chroot $FILESYSTEM_ROOT apt-get update


apt-get update

No need to update again. #Closed

apt-get update

No need to update again.

Removed

qiluo-msft · 2024-01-30T22:18:08Z

build_debian.sh

    sudo https_proxy=$https_proxy LANG=C chroot $FILESYSTEM_ROOT curl -o /tmp/cri-dockerd.deb -fsSL \
-        https://github.com/Mirantis/cri-dockerd/releases/download/v${MASTER_CRI_DOCKERD}/cri-dockerd_${MASTER_CRI_DOCKERD}.3-0.debian-${IMAGE_DISTRO}_amd64.deb
+        https://github.com/Mirantis/cri-dockerd/releases/download/v${MASTER_CRI_DOCKERD}/cri-dockerd_${MASTER_CRI_DOCKERD}.3-0.debian-bullseye_amd64.deb


bullseye

Fixed distro is not future-proof. #Closed

bullseye

Fixed distro is not future-proof.

Have changed this back, the cri-dockerd latest version has bookworm version, I have upgraded the version to 0.3.10

qiluo-msft · 2024-01-30T22:19:46Z

build_debian.sh

@@ -271,6 +271,8 @@ sudo LANG=C chroot $FILESYSTEM_ROOT apt-get -y install docker-ce=${DOCKER_VERSIO

 install_kubernetes () {
    local ver="$1"
+    sudo LANG=C chroot $FILESYSTEM_ROOT apt-get update
+    sudo LANG=C chroot $FILESYSTEM_ROOT apt-get -y install gnupg


gnupg

This will install more package(s) into sonic image. Is it really needed? Your intention is to change master image only.

gnupg

This will install more package(s) into sonic image. Is it really needed? Your intention is to change master image only.

Yes, indeed need to install gnupg before install gpg file for bookworm, I have removed gnupg after installing the gpg file.

lixiaoyuner · 2024-02-01T02:57:00Z

/azpw run

mssonicbld · 2024-02-01T02:57:02Z

/AzurePipelines run

azure-pipelines · 2024-02-01T02:57:07Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

lixiaoyuner · 2024-02-18T05:46:20Z

/azpw run

mssonicbld · 2024-02-18T05:46:22Z

/AzurePipelines run

azure-pipelines · 2024-02-18T05:46:28Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

liushilongbuaa · 2024-02-19T09:36:53Z

/azpw ms_conflict

liushilongbuaa · 2024-02-20T02:18:09Z

/azpw ms_conflict

qiluo-msft · 2024-02-21T01:05:35Z

build_debian.sh

    sudo https_proxy=$https_proxy LANG=C chroot $FILESYSTEM_ROOT curl -fsSL \
        https://packages.cloud.google.com/apt/doc/apt-key.gpg | \
        sudo LANG=C chroot $FILESYSTEM_ROOT apt-key add -
+    sudo LANG=C chroot $FILESYSTEM_ROOT apt-get -y remove gnupg


remove

purge is better than remove #Closed

qiluo-msft · 2024-02-21T02:24:32Z

files/image_config/kubernetes/kubernetes_master_entrance.sh

@@ -2,7 +2,7 @@

 # This script is for kubernetes master image usage
 # Will mount kubernetes master disk and execute kubernetes entrance script
-
+#


Unnecessary change?

Install azure keyvault python package

3bc984e

Signed-off-by: Yun Li <[email protected]>

lixiaoyuner requested a review from lguohan as a code owner January 17, 2024 08:24

lixiaoyuner added 8 commits January 18, 2024 13:03

Need to install gnupg before add gpg file

834b8b3

Fix the extra quote error

9f46b01

The netcat package has changed to netcat-openbsd

e9fa29b

Remove the debug output

2078cac

Install bullseye version for test

e691a31

Generate a vhdx disk

c9e2f27

Rename the vhdx disk name

d5dae12

Remove the extra comment line

9b69890

xumia reviewed Jan 25, 2024

View reviewed changes

xumia previously approved these changes Jan 25, 2024

View reviewed changes

Remove the unnecessary param k for gzip

e65bae4

lixiaoyuner dismissed xumia’s stale review via e65bae4 January 25, 2024 06:57

xumia previously approved these changes Jan 25, 2024

View reviewed changes

losha228 approved these changes Jan 30, 2024

View reviewed changes

qiluo-msft reviewed Jan 30, 2024

View reviewed changes

Fix comments

32b628a

lixiaoyuner dismissed xumia’s stale review via 32b628a January 31, 2024 08:04

lixiaoyuner added 4 commits February 1, 2024 12:35

Fix k8s master flag unbale to add issue

f7d70dd

Use current skipvstest yaml for test

2bb5595

Fix the include_kubernetes_master flag issue

6e911db

Fix

ba11f72

lixiaoyuner added 6 commits February 1, 2024 18:42

Remove x param to test

1dc8645

Update the k8s change flag variable name

21ca2cb

Merge branch 'master' into dev/yunli1/install-azure-keyvault-pakcage

0f2a661

Make the CRI-dockerd installation clear

942d218

Remove local due to it is a functional flag

a84c6b2

Change the skip template back

fb07bb7

lixiaoyuner added 4 commits February 18, 2024 13:56

Retrigger the build

7a00c31

Upgrade the cri-dockerd to 0.3.10

b3bd9ca

Merge branch 'master' into dev/yunli1/install-azure-keyvault-pakcage

336065f

Change the installing cri-dockerd logic back

ef2fca3

lixiaoyuner added 2 commits February 19, 2024 18:16

Revert the master script change

a661a64

Change the skip vs test template back

f61ee84

liushilongbuaa approved these changes Feb 20, 2024

View reviewed changes

qiluo-msft reviewed Feb 21, 2024

View reviewed changes

Fix comments

0e653f1

qiluo-msft reviewed Feb 21, 2024

View reviewed changes

lixiaoyuner added 3 commits February 21, 2024 16:35

Reproduce the error case

a97b372

Update

6cf4552

Remove the unnecessary comment

ba0c24d

		@@ -299,10 +301,9 @@ then
		install_kubernetes ${MASTER_KUBERNETES_VERSION}

		sudo LANG=C chroot $FILESYSTEM_ROOT apt-get update

Install azure keyvault python package for k8s master image #17806

Are you sure you want to change the base?

Install azure keyvault python package for k8s master image #17806

Uh oh!

Conversation

lixiaoyuner commented Jan 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xumia left a comment

Choose a reason for hiding this comment

Uh oh!

losha228 left a comment

Choose a reason for hiding this comment

Uh oh!

qiluo-msft Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qiluo-msft commented Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qiluo-msft Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qiluo-msft Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lixiaoyuner commented Feb 1, 2024

Uh oh!

mssonicbld commented Feb 1, 2024

Uh oh!

azure-pipelines bot commented Feb 1, 2024

Uh oh!

lixiaoyuner commented Feb 18, 2024

Uh oh!

mssonicbld commented Feb 18, 2024

Uh oh!

azure-pipelines bot commented Feb 18, 2024

Uh oh!

liushilongbuaa commented Feb 19, 2024

Uh oh!

liushilongbuaa commented Feb 20, 2024

Uh oh!

qiluo-msft Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lixiaoyuner commented Jan 17, 2024 •

edited

Loading

qiluo-msft Jan 30, 2024 •

edited

Loading

qiluo-msft commented Jan 30, 2024 •

edited

Loading

qiluo-msft Jan 30, 2024 •

edited

Loading

qiluo-msft Jan 30, 2024 •

edited

Loading

qiluo-msft Feb 21, 2024 •

edited

Loading