feat(k8s): improve k8s scanning to handle namespace-restricted controllers

[afdesk]

Description

By design Trivy k8s scans only targeted control plane components (Infra Assessment), which made sense for cluster-level scanning (without namespaces).

However, Trivy now supports scanning controllers that may be restricted to specific namespaces.

We should enhance the handling of such cases to ensure accurate and comprehensive scanning.

See below for details:

Without the --include-namespaces flag

➜  ingress-nightmare ~/repos/trivy/trivy.new k8s  --scanners=vuln --report summary
2025-03-26T15:04:44-06:00	INFO	Node scanning is enabled
2025-03-26T15:04:44-06:00	INFO	If you want to disable Node scanning via an in-cluster Job, please try '--disable-node-collector' to disable the Node-Collector job.
2025-03-26T15:04:44-06:00	INFO	Scanning K8s...	K8s="kind-kind-cluster"
248 / 248 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 22 p/s

Summary Report for kind-kind-cluster


Workload Assessment
┌────────────────────┬─────────────────────────────────────────────┬──────────────────────┐
│     Namespace      │                  Resource                   │   Vulnerabilities    │
│                    │                                             ├───┬────┬────┬────┬───┤
│                    │                                             │ C │ H  │ M  │ L  │ U │
├────────────────────┼─────────────────────────────────────────────┼───┼────┼────┼────┼───┤
│ projectcontour     │ Job/contour-certgen-v1-30-1                 │   │    │ 4  │    │   │
│ projectcontour     │ Deployment/contour                          │   │    │ 4  │    │   │
│ projectcontour     │ DaemonSet/envoy                             │   │    │ 25 │ 44 │   │
│ local-path-storage │ Deployment/local-path-provisioner           │ 1 │    │ 17 │ 11 │ 2 │
│ ingress-nginx      │ Job/ingress-nginx-admission-patch           │ 1 │ 2  │ 9  │    │ 2 │
│ ingress-nginx      │ Deployment/ingress-nginx-controller         │   │ 11 │ 38 │ 7  │ 2 │
│ ingress-nginx      │ ControlPlaneComponents/k8s.io/ingress-nginx │ 1 │ 4  │ 1  │    │   │
│ ingress-nginx      │ Job/ingress-nginx-admission-create          │ 1 │ 2  │ 9  │    │ 2 │
└────────────────────┴─────────────────────────────────────────────┴───┴────┴────┴────┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌─────────────┬────────────────────────────────────────────────────────┬─────────────────────┐
│  Namespace  │                        Resource                        │   Vulnerabilities   │
│             │                                                        ├───┬───┬────┬────┬───┤
│             │                                                        │ C │ H │ M  │ L  │ U │
├─────────────┼────────────────────────────────────────────────────────┼───┼───┼────┼────┼───┤
│ kube-system │ Deployment/coredns                                     │ 1 │ 2 │ 8  │ 1  │ 2 │
│ kube-system │ Pod/kube-apiserver-kind-cluster-control-plane          │ 1 │   │ 8  │    │   │
│ kube-system │ Pod/kube-controller-manager-kind-cluster-control-plane │ 1 │   │ 8  │    │   │
│ kube-system │ Pod/kube-scheduler-kind-cluster-control-plane          │ 1 │   │ 7  │    │   │
│ kube-system │ DaemonSet/kindnet                                      │   │ 1 │ 20 │ 16 │   │
│ kube-system │ DaemonSet/kube-proxy                                   │ 1 │   │ 9  │ 14 │   │
│ kube-system │ Pod/etcd-kind-cluster-control-plane                    │ 4 │ 4 │ 19 │ 4  │   │
│             │ Node/kind-cluster-control-plane                        │   │   │ 1  │    │   │
└─────────────┴────────────────────────────────────────────────────────┴───┴───┴────┴────┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

With the --include-namespaces flag

➜  ingress-nightmare ~/repos/trivy/trivy.new k8s  --scanners=vuln --report summary --include-namespaces=ingress-nginx
2025-03-26T15:04:35-06:00	INFO	Node scanning is enabled
2025-03-26T15:04:35-06:00	INFO	If you want to disable Node scanning via an in-cluster Job, please try '--disable-node-collector' to disable the Node-Collector job.
2025-03-26T15:04:35-06:00	INFO	Scanning K8s...	K8s="kind-kind-cluster"
14 / 14 [-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 8 p/s

Summary Report for kind-kind-cluster


Workload Assessment
┌───────────────┬─────────────────────────────────────┬─────────────────────┐
│   Namespace   │              Resource               │   Vulnerabilities   │
│               │                                     ├───┬────┬────┬───┬───┤
│               │                                     │ C │ H  │ M  │ L │ U │
├───────────────┼─────────────────────────────────────┼───┼────┼────┼───┼───┤
│ ingress-nginx │ Job/ingress-nginx-admission-patch   │ 1 │ 2  │ 9  │   │ 2 │
│ ingress-nginx │ Deployment/ingress-nginx-controller │   │ 11 │ 38 │ 7 │ 2 │
│ ingress-nginx │ Job/ingress-nginx-admission-create  │ 1 │ 2  │ 9  │   │ 2 │
└───────────────┴─────────────────────────────────────┴───┴────┴────┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌───────────┬──────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │
│           │          ├───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Observe that ControlPlaneComponents/k8s.io/ingress-nginx does not show up when filtering by namespace even though it is in the same namespace as the filter.

Originally posted by @simar7 in #8614 (comment)

[afdesk]

additional info. it comes from here:
https://github.com/aquasecurity/trivy-kubernetes/blob/f0d867cade215d1ac88b8d5d66650c00d56a748c/pkg/trivyk8s/trivyk8s.go#L300-306

also we should skip node checks here:

trivy/pkg/k8s/scanner/scanner.go

Lines 228 to 231 in ad58cf4

	var nodeName string
	if nodeName = s.findNodeName(artifactsData); nodeName == "" {
	return resources, nil
	}

trivy/pkg/k8s/scanner/scanner.go

Lines 377 to 381 in ad58cf4

	// Find the first node name to identify AKS cluster
	var nodeName string
	if nodeName = s.findNodeName(allArtifact); nodeName == "" {
	return nil, errors.New("failed to find node name")
	}

[afdesk]

I also tested scanning for kube-system namespace:

$ trivy k8s --report summary
...


Infra Assessment
┌─────────────┬──────────────────────────────────────────────┬──────────────────────┬────────────────────┬───────────────────┐
│  Namespace  │                   Resource                   │   Vulnerabilities    │ Misconfigurations  │      Secrets      │
│             │                                              ├───┬────┬────┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│             │                                              │ C │ H  │ M  │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├─────────────┼──────────────────────────────────────────────┼───┼────┼────┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ kube-system │ Pod/kube-apiserver-minikube                  │ 1 │ 2  │ 13 │    │   │   │ 2 │ 7 │ 19 │   │   │   │   │   │   │
│ kube-system │ Pod/kube-controller-manager-minikube         │ 1 │ 2  │ 13 │    │   │   │ 2 │ 6 │ 12 │   │   │   │   │   │   │
│ kube-system │ Pod/kube-scheduler-minikube                  │ 1 │ 2  │ 12 │    │   │   │ 2 │ 6 │ 10 │   │   │   │   │   │   │
│ kube-system │ ConfigMap/extension-apiserver-authentication │   │    │    │    │   │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system │ DaemonSet/kube-proxy                         │ 1 │ 2  │ 18 │ 16 │   │   │ 3 │ 8 │ 12 │   │   │   │   │   │   │
│ kube-system │ Deployment/coredns                           │ 2 │ 8  │ 24 │ 1  │ 2 │   │   │ 8 │ 6  │   │   │   │   │   │   │
│ kube-system │ Pod/storage-provisioner                      │ 5 │ 52 │ 38 │ 1  │   │   │ 3 │ 7 │ 12 │   │   │   │   │   │   │
│ kube-system │ Service/kube-dns                             │   │    │    │    │   │   │   │ 1 │ 2  │   │   │   │   │   │   │
│ kube-system │ Pod/etcd-minikube                            │ 4 │ 9  │ 29 │ 4  │   │   │ 2 │ 6 │ 9  │   │   │   │   │   │   │
│             │ Node/minikube                                │   │    │    │    │   │   │ 3 │   │ 2  │   │   │   │   │   │   │
└─────────────┴──────────────────────────────────────────────┴───┴────┴────┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

$ trivy k8s --report summary --include-namespaces kube-system
...
Infra Assessment
┌─────────────┬──────────────────────────────────────────────┬──────────────────────┬────────────────────┬───────────────────┐
│  Namespace  │                   Resource                   │   Vulnerabilities    │ Misconfigurations  │      Secrets      │
│             │                                              ├───┬────┬────┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│             │                                              │ C │ H  │ M  │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├─────────────┼──────────────────────────────────────────────┼───┼────┼────┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ kube-system │ ConfigMap/extension-apiserver-authentication │   │    │    │    │   │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system │ Deployment/coredns                           │ 2 │ 8  │ 24 │ 1  │ 2 │   │   │ 8 │ 6  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-apiserver-minikube                  │ 1 │ 2  │ 13 │    │   │   │ 2 │ 7 │ 19 │   │   │   │   │   │   │
│ kube-system │ Pod/kube-scheduler-minikube                  │ 1 │ 2  │ 12 │    │   │   │ 2 │ 6 │ 10 │   │   │   │   │   │   │
│ kube-system │ Service/kube-dns                             │   │    │    │    │   │   │   │ 1 │ 2  │   │   │   │   │   │   │
│ kube-system │ DaemonSet/kube-proxy                         │ 1 │ 2  │ 18 │ 16 │   │   │ 3 │ 8 │ 12 │   │   │   │   │   │   │
│ kube-system │ Pod/etcd-minikube                            │ 4 │ 9  │ 29 │ 4  │   │   │ 2 │ 6 │ 9  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-controller-manager-minikube         │ 1 │ 2  │ 13 │    │   │   │ 2 │ 6 │ 12 │   │   │   │   │   │   │
│ kube-system │ Pod/storage-provisioner                      │ 5 │ 52 │ 38 │ 1  │   │   │ 3 │ 7 │ 12 │   │   │   │   │   │   │
└─────────────┴──────────────────────────────────────────────┴───┴────┴────┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

[itaysk]

isn't this a bug?

[afdesk]

isn't this a bug?

I think that no,
it was done on purpose: aquasecurity/trivy-kubernetes#254

[simar7]

Closed via #8918