Print table failed due to not supported less than compare between string and IPAddress #469

aljoshare · 2021-07-16T12:34:38Z

What happened

The following error happens when running kube-hunter with the following definition on k8s v1.21.2:

containers:
            - name: kube-hunter
              image: aquasec/kube-hunter:0.5.2
              command: [ "kube-hunter" ]
              args: [ "--pod", "--log", "DEBUG" ]

Expected behavior

The table should be generated and printed to stdout

The text was updated successfully, but these errors were encountered:

danielsagi · 2021-07-22T12:25:14Z

Thanks for reporting!
This looks really weird. could you provide the full debug log data? @AljoschaP

julien-sugg · 2021-07-22T13:49:03Z

Hi,

I started playing with kube-hunter today with my playground cluster on GKE and I am facing a similar issue when I create a dedicated NetworkPolicy for the Pod.

I will create a separate issue if @AljoschaP 's issue is not exactly the same as the one I am facing.

Current Setup

Versions

kube-hunter: v0.5.2 built from the Dockerfile in the repository for tag 0.5.2
kubernetes: v1.19.10-gke.1600 (Server) and v1.19.4 (Client)

Manifests to reproduce

The simplified manifests below are sufficient to reproduce the issue and non-cluster resources will be located in the default namespace.

Note that, for brievety, the NetworkPolicy applies to the whole namespace (podSelector: {})

NetworkPolicies are baked by Calico CNI plugin.

# kube-hunter.yaml
---
# SA
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ci-kube-hunter-in-cluster
  namespace: default
---
# ClusterRole
# FIXME, kube-hunter project doesn't seem provide an exhaustive list of the required
# RBAC permissions, so we simply grant it read access (get, list) to all resources for now
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ci-kube-hunter-in-cluster
rules:
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: ["get", "list"]
---
# Bind SA to the ClusterRole.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ci-kube-hunter-in-cluster
subjects:
  - kind: ServiceAccount
    name: ci-kube-hunter-in-cluster
    namespace: default
    apiGroup: ""
roleRef:
  kind: ClusterRole
  name: ci-kube-hunter-in-cluster
  apiGroup: rbac.authorization.k8s.io
---
# NetworkPolicy
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: ci-kube-hunter-in-cluster
  namespace: default
spec:
  policyTypes:
    - Egress
    - Ingress
  podSelector: {} # simplified to match all pods for brievety
  egress:
    # Allow DNS
    - to:
        - podSelector:
            matchLabels:
              k8s-app: kube-dns
          namespaceSelector:
            matchLabels:
              networking/namespace: kube-system
      ports:
        - protocol: TCP
          port: 53
        - protocol: UDP
          port: 53
    # Allow Kubernetes networks
    # TODO, need to check in details what kube-hunter really needs to access
    - to:
        - ipBlock:
            # Master Network (kube api server)
            cidr: *redacted*
        - ipBlock:
            # Nodes Networks
            cidr: *redacted*
        - ipBlock:
            # Pods IPs
            cidr: *redacted*
        - ipBlock:
            # Service IPs
            cidr: *redacted*
---
apiVersion: batch/v1
kind: Job
metadata:
  name: ci-kube-hunter-in-cluster
  namespace: default
spec:
  template:
    spec:
      serviceAccountName: ci-kube-hunter-in-cluster
      containers:
        - name: kube-hunter-in-cluster
          # @see https://github.com/aquasecurity/kube-hunter/tree/main#container
          image: *redacted*/kube-hunter:0.5.2 # ${KUBE_HUNTER_VERSION}
          command: ["kube-hunter"]
          args: ["--pod", "--report", "plain", "--log", "debug"]
      restartPolicy: Never

k apply -f kube-hunter.yaml

Logs

The logs below were generated from a kube-hunter run on a freshly generated cluster with no additional workloads.

Note that I manually redacted the auth_token and client_cert info using a regexp

# The output is too long for GitHub comments

See output.txt

Note that when I remove the NetworkPolicy, or more precisely perform an allow all on egress (.spec.egress[0].to: {}), the error doesn't occur any more and the report is properly available in stdout.

Optionally, can you please clarify, if available, the exhaustive list of required RBAC permissions and network egress targets ?

Signed-off-by: Daniel Pacak <[email protected]>

aljoshare · 2021-09-04T15:38:36Z

Sorry for the late reply. I think the issue of @julien-sugg is the same as the one that occurs in our cluster.

Alevsk · 2023-09-23T07:17:25Z

Same issue is happening to me when running kube-hunter inside the cluster using the --pod flag

Kube-hunter deployment

apiVersion: v1
kind: Pod
metadata:
  name: kube-hunter-with-sa
  namespace: default
  labels:
    app: kube-hunter-with-sa
spec:
  serviceAccountName: kube-hunter
  containers:
    - name: kube-hunter-with-sa
      image: aquasec/kube-hunter:0.6.8
      command: ["kube-hunter"]
      args: ["--pod",  "--log", "debug"]
  restartPolicy: Never

Kubernetes version:

NAME                             STATUS   ROLES           AGE    VERSION
workshop-cluster-control-plane   Ready    control-plane   3d3h   v1.27.3

Full error log

2023-09-23 07:13:24,403 DEBUG kube_hunter.core.events.event_handler Event <class 'kube_hunter.core.events.types.HuntFinished'> got published to hunter - <class 'kube_hunter.modules.report. │
│ 2023-09-23 07:13:24,403 DEBUG kube_hunter.core.events.event_handler Executing <class 'kube_hunter.modules.report.collector.SendFullReport'> with {'previous': None, 'hunter': None}          │
│ 2023-09-23 07:13:24,406 DEBUG kube_hunter.core.events.event_handler '<' not supported between instances of 'str' and 'IPAddress'                                                             │
│ Traceback (most recent call last):                                                                                                                                                           │
│   File "/usr/local/lib/python3.8/site-packages/kube_hunter/core/events/event_handler.py", line 346, in worker                                                                                │
│     hook.execute()                                                                                                                                                                           │
│   File "/usr/local/lib/python3.8/site-packages/kube_hunter/modules/report/collector.py", line 56, in execute                                                                                 │
│     report = config.reporter.get_report(statistics=config.statistics, mapping=config.mapping)                                                                                                │
│   File "/usr/local/lib/python3.8/site-packages/kube_hunter/modules/report/plain.py", line 30, in get_report                                                                                  │
│     output += self.nodes_table()                                                                                                                                                             │
│   File "/usr/local/lib/python3.8/site-packages/kube_hunter/modules/report/plain.py", line 62, in nodes_table                                                                                 │
│     nodes_ret = f"\nNodes\n{nodes_table}\n"                                                                                                                                                  │
│   File "/usr/local/lib/python3.8/site-packages/prettytable/prettytable.py", line 327, in __str__                                                                                             │
│     return self.get_string()                                                                                                                                                                 │
│   File "/usr/local/lib/python3.8/site-packages/prettytable/prettytable.py", line 1700, in get_string                                                                                         │
│     rows = self._get_rows(options)                                                                                                                                                           │
│   File "/usr/local/lib/python3.8/site-packages/prettytable/prettytable.py", line 1618, in _get_rows                                                                                          │
│     rows.sort(reverse=options["reversesort"], key=options["sort_key"])                                                                                                                       │
│ TypeError: '<' not supported between instances of 'str' and 'IPAddress'                                                                                                                      │
│ 2023-09-23 07:13:24,412 DEBUG kube_hunter.__main__ Cleaned Queue

aljoshare added the bug Something isn't working label Jul 16, 2021

py-go pushed a commit to py-go/kube-hunter that referenced this issue Aug 29, 2021

refactor: Use client.Client in integration test (aquasecurity#469)

fa27379

Signed-off-by: Daniel Pacak <[email protected]>

mckirk linked a pull request Dec 9, 2022 that will close this issue

Fix: Convert host to str before adding to node_tables (#469) #530

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Print table failed due to not supported less than compare between string and IPAddress #469

Print table failed due to not supported less than compare between string and IPAddress #469

aljoshare commented Jul 16, 2021

danielsagi commented Jul 22, 2021

julien-sugg commented Jul 22, 2021 •

edited

Loading

aljoshare commented Sep 4, 2021

Alevsk commented Sep 23, 2023

Print table failed due to not supported less than compare between string and IPAddress #469

Print table failed due to not supported less than compare between string and IPAddress #469

Comments

aljoshare commented Jul 16, 2021

What happened

Expected behavior

danielsagi commented Jul 22, 2021

julien-sugg commented Jul 22, 2021 • edited Loading

Current Setup

Versions

Manifests to reproduce

Logs

aljoshare commented Sep 4, 2021

Alevsk commented Sep 23, 2023

julien-sugg commented Jul 22, 2021 •

edited

Loading