[BUG] Missing requests/limits for AKS core components on worker nodes #3496

slawekww · 2023-02-28T07:43:22Z

Describe the bug
Run https://github.com/Azure/kube-advisor on AKS 1.24.6
to verify CPU/Memory requests/limits set for k8s workloads.

The following components provided and maintain by AKS on worker nodes have no requests/limits set:

calico-kube-controllers
calico-node
calico-typha
cloud-node-manager
csi-azuredisk-node
csi-azurefile-node
kube-proxy
tigera-operator

To Reproduce
Steps to reproduce the behavior:

https://github.com/Azure/kube-advisor#running-in-a-kubernetes-cluster-with-rbac-enabled
See output on AKS cluster

Expected behavior
All AKS managed workloads on worker nodes should have limits/request set and have guidelines to update them by AKS administrators.

Screenshots
kube-advisory.log

Environment (please complete the following information):

CLI Version: 2.44.1
Kubernetes version: AKS 1.24.6
CLI Extension version: n/a
Browser: n/a

Additional context
Cost of worker node is under customers and it should be clear and precise way to control AKS managed workloads.

marcindulak · 2023-03-06T16:57:28Z

Discussed previously in #2125 (comment)

slawekww · 2023-03-08T08:40:00Z

@marcindulak Thanks!
Discussion is about OOMKill for AKS core components.

Once they would have Guaranteed QoS class https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/#create-a-pod-that-gets-assigned-a-qos-class-of-guaranteed
it would not be issue with OOMKill as they would requests=limits the expected resources.

AKS administrators should have possibility to update requests/limits for those AKS core components.
In case of avoiding OOMKill, use Guaranteed QoS class.
In case of cost optimization, set values which are acceptable by customers - we need to know minimum values for requests CPU/memory for AKS core components.

ghost · 2023-05-07T01:00:52Z

Action required from @Azure/aks-pm

ghost · 2023-05-22T06:00:56Z