HNC: excluded-namespace and included-namespace

[santinoncs]

Hi,

I want to use HNC but only in a few specific namespaces. Because of that, it is far more difficult to exclude-namespaces and include few than the other way.
Also it would be great if you can use regexp
Do you think is feasible ? it makes sense?
this would be like

    - args:
      - --webhook-server-port=9443
      - --metrics-addr=:8080
      - --max-reconciles=10
      - --apiserver-qps-throttle=50
      - --enable-internal-cert-management
      - --include-namespaces="tests-.*"

[adrianludwin]

Can you explain what you mean by "use HNC... in a few specific namespaces?" By and large, if you don't put any HNC configs into a namespace, HNC will ignore it, *unless* something goes terribly wrong with HNC. --exclude-namespaces exists to protect the most critical namespaces on your cluster from HNC going nuts; it's not really a user-visible feature, so much as it's a way to make sure that no matter how badly HNC behaves, the rest of the cluster is operating well enough so that (say) you can delete HNC. It's also important to understand that --exclude-namespaces on its own as actually just a backup feature that operates on *non* excluded namespaces. Let me explain: HNC protects critical system namespaces (like kube-system) by having its webhook include a namespace selector <https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector> that excludes all namespaces with the label hnc.x-k8s.io/excluded-namespace (unfortunately, webhooks don't let you exclude namespaces by name). But this means that anyone who can edit a namespace could just *add* this label to their own namespace, and HNC would start to ignore it. Since HNC is part of the security system of your entire cluster, we shouldn't allow this, and so if HNC sees that someone's added the label to any namespace, it automatically removes it - *unless* that namespace is part of --exclude-namespaces. With that said, I can see a good reason to implement this feature - if you have very large clusters, and you don't trust HNC yet (a very reasonable position), but want to use it in some areas. If that matches what you're trying to do, then I'd totally support it if someone wanted to make a PR, and I'll help review it. Here's how I'd approach this problem: - Modify the webhook (here <https://github.com/kubernetes-sigs/hierarchical-namespaces/blob/ba15870be8ba46b259891c1cd55634e7e438d62c/config/webhook/webhook_patch.yaml>) so that instead of *excluding* all namespaces with hnc.x-k8s.io/excluded-namespace, it *ignores* all namespaces *without* the label hnc.x-k8s.io/included-namespace (or something similar). - Find all places in the code that examine the ExcludedNamespaces var <https://github.com/kubernetes-sigs/hierarchical-namespaces/blob/ba15870be8ba46b259891c1cd55634e7e438d62c/internal/config/default_config.go#L17>, and replace them with a function that does the regex match instead - Modify HNC to automatically add this label to all namespaces that are covered by --included-namespace, but that are *not* covered by --excluded-namespace Also, I might call the flag --included-namespaces-regex to disambiguate its behaviour from the existing --excluded-namespace, which can be specified multiple times and is definitely *not* a regex. I hope this helps! Thanks, A

…

On Thu, Sep 30, 2021 at 8:17 AM Santiago Nuñez-Cacho < ***@***.***> wrote: Hi, I want to use HNC but only in a few specific namespaces. Because of that, it is far more difficult to exclude-namespaces and include few than the other way. Also it would be great if you can use regexp Do you think is feasible ? it makes sense? this would be like - args: - --webhook-server-port=9443 - --metrics-addr=:8080 - --max-reconciles=10 - --apiserver-qps-throttle=50 - --enable-internal-cert-management - --include-namespaces="tests-.*" — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#87>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE43PZFROT4XN3HNS5NHHNLUERIPDANCNFSM5FCGEGNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[santinoncs]

Problem is , the controller pod was not running and this was the cause that all the operation of Create and Delete were not working

Error from server (InternalError): error when deleting "/tmp/nginx.yaml": Internal error occurred: failed calling webhook "objects.hnc.x-k8s.io": Post "https://hnc-webhook-service.hnc-system.svc:443/validate-objects?timeout=2s": no endpoints available for service "hnc-webhook-service"

If I could prevent this to happen in the most of the namespaces, that would be great.

this is why i wanted this in the first place.

Thanks a lot for your rapid response. I will take a look at your approach.

[adrianludwin]

Great, thanks!

…

On Thu, Sep 30, 2021 at 9:14 AM Santiago Nuñez-Cacho < ***@***.***> wrote: Problem is , the controller pod was not running and this was the cause that all the operation of Create and Delete were not working Error from server (InternalError): error when deleting "/tmp/nginx.yaml": Internal error occurred: failed calling webhook "objects.hnc.x-k8s.io": Post " https://hnc-webhook-service.hnc-system.svc:443/validate-objects?timeout=2s": no endpoints available for service "hnc-webhook-service" If I could prevent this to happen in the most of the namespaces, that would be great. this is why i wanted this in the first place. Thanks a lot for your rapid response. I will take a look at your approach. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#87 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE43PZGXQMKZ3Y5LAURMAHTUERPEXANCNFSM5FCGEGNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[santinoncs]

some insights about the implementation:

• As for now the operator will treat as included everything that is not in the --exclude-namespaces list

What will happen if we use --included-namespaces?

should from now on treat as included just the namespaces that are in that list and excluded the ones that are in the excluded list?

One approach could be:

• With these start operator parameters

--excluded-namespaces=kube-system

this would mean, everything not in this list will be included

• With these parameters:

--included-namespaces=test

this would mean everything except test will be excluded

does this make sense?

[adrianludwin]

I think the most typical way to combine "included" and "excluded" options are: "everything in 'included', unless also in 'excluded'." That lets you do things like:

--included-namespaces="^test.*"
--excluded-namespace="test-system"

That is, include every namespace that starts with test other than test-system.

[santinoncs]

What about if only one of the two ( included / excluded ) exists?

I imagine that the logic would be, if only one exists, just everything else is not.

[adrianludwin]

The regex to exclude system namespaces would be... unfortunate. Technically it's possible but I think it would be sufficiently hard to use that I'd rather keep --excluded-namespace the way it is.