[Release-1.32] - Cannot enable 1.32 feature gates: agent kubelet emulated version 1.31

[brandond]

Backport fix for Cannot enable 1.32 feature gates: agent kubelet emulated version 1.31

• Cannot enable 1.32 feature gates: agent kubelet emulated version 1.31 #11937

[VestigeJ]

##Environment Details
Validated using COMMIT=6546ba9a4aad2f5a51c2259be3a9a9f9f1739e06

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

Linux 6.4.0-150600.23.17-default x86_64 GNU/Linux
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"

Cluster Configuration:

NAME               STATUS   ROLES                  AGE   VERSION
ip-1-1-1-86        Ready    <none>                 20m   v1.32.3+k3s-6546ba9a
ip-1-1-1-28        Ready    control-plane,master   21m   v1.32.3+k3s-6546ba9a

Config.yaml:

Server config

node-external-ip: 1.1.1.1
node-ip: 1.1.1.28
write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
kube-apiserver-arg:
- --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true
kube-controller-manager-arg:
- --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true
kube-proxy-arg:
- --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true
kube-scheduler-arg:
- --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true
kubelet-arg:
- --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true

Agent config

=========== k3s config ===========
node-external-ip: 1.1.1.2
node-ip: 1.1.1.86
server: https://1.1.1.2:6443
token: YOUR_TOKEN_HERE
kubelet-arg:
  - --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true
kube-proxy-arg:
  - --feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true

Validation Steps

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf
$ sudo cp 90-kubelet.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_K3S_COMMIT=6546ba9a4aad2f5a51c2259be3a9a9f9f1739e06 INSTALL_K3S_EXEC=server ./install-k3s.sh
$ w2 kg no,po,svc -A
$ kgn -o yaml | grep k3s.io/node-
$ k3s -v

Results:

$ kgn -o yaml | grep k3s.io/node-

k3s.io/node-args: '["agent","--node-external-ip","1.1.1.1","--node-ip","1.1.1.86","--server","https://1.1.1.1:6443","--token","************","--kubelet-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true","--kube-proxy-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true"]'
      k3s.io/node-config-hash: 43TDANCE1234FIGHT4DA3A====
      k3s.io/node-env: '{}'
      k3s.io/node-args: '["server","--node-external-ip","1.1.1.2","--node-ip","1.1.1.28","--write-kubeconfig-mode","644","--debug","true","--token","************","--kube-apiserver-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true","--kube-controller-manager-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true","--kube-proxy-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true","--kube-scheduler-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true","--kubelet-arg","--feature-gates=InPlacePodVerticalScaling=true,InPlacePodVerticalScalingAllocatedStatus=true,InPlacePodVerticalScalingExclusiveCPUs=true"]'
      k3s.io/node-config-hash: 43TDANCE1234FIGHT4DA3A====
      k3s.io/node-env: '{}'