Document system:authenticated group usage in GKE

[alemorcuq]

We are binding the system:authenticated group to the service-proxier role so that kubeseal can easily communicate with the controller to get the public key and encrypt your secrets. This role only has permissions to get the sealed-secrets service and to make POST and GET requests to that service.

In GKE, the use of system:authenticated is discouraged because anyone with a Google account is assigned to that group. Here's a recent KubeCon talk on this topic that mentions our use case.

While nothing sensitive is exposed to users on that group, we should document this in our GKE documention. We should also add an option in the chart to allow users to disable the creation of the service-proxier service.

[keithjpaulson]

use of system:authenticated is more than discouraged; from google docs
"To help secure your clusters against mass malware attacks that exploit cluster-admin access misconfigurations, GKE clusters running version 1.28 and later won't allow you to bind the cluster-admin ClusterRole to the system:anonymous user or to the system:unauthenticated or system:authenticated groups."
I'd guess this will cause issues on GKE if the service-proxier isn't disabled, or group isn't changed?

[alemorcuq]

@keithjpaulson:

From your quote, it says that you won't be able to bind the cluster-admin ClusterRole to system:authenticated. Doing this effectively gives full admin rights to your cluster to anyone with a Google account, that's why Google is disabling it.

We are not doing that. We are binding a custom Role that only have permissions to do GET and POST requests to a specific service created by us.

[keithjpaulson]

@alemorcuq Correct, I read that part wrong -- they will just flag any use of system:authenticated as potentially insecure in console popups and emails that make administrators like me (who haven't touched sealed secrets) panic a bit. Apologies.

[cormacrelf]

On GKE 1.32.2-gke.1182003, GKE Warden now completely rejects the use of system:authenticated. No longer able to install sealed-secrets (with the default configuration, haven't tried using #1451). Here's the error from Flux's helm controller.

Helm install failed for release sealed-secrets/sealed-secrets-controller with chart sealed-secrets@2.17.2: 1 error occurred: * admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints. Violations details: {"[denied by rbac-binding-limitation]":["Binding any Role or ClusterRole to Group "system:authenticated" is forbidden."]} Requested by user: 'system:serviceaccount:flux-system:helm-controller', groups: 'system:serviceaccounts,system:serviceaccounts:flux-system,system:authenticated'.

[cormacrelf]

For those using GKE and also Google Workspace (or Google Cloud Identity, the free one without Gmail etc), this serves as an alternative:

1. Set up Google Groups for RBAC. That means a group named exactly gke-security-groups@yourdomain.com as the anchor, and then subgroups.¹

2. You tell your cluster to map all groups that are members of that anchor to a k8s Group in RBAC. Follow the docs from step 1, or the terraform below.

3. Set helm values for sealed-secrets to one of these groups.

   # values.yaml, or in a flux HelmRelease `values:` section
   rbac:
     serviceProxier:
       subjects:
         - apiGroup: rbac.authorization.k8s.io
           kind: Group
           name: gke-clustername-prod-secrets-team@yourdomain.com

4. Go edit your groups (not the anchor, but the subgroups) at groups.google.com and give out access to your team members.

Terraform implementation

For step 1, the following Terraform is what the docs describe. For the manual steps in NOTE: blocks, the "who can view members = group members" settings seems to be the default for me, YMMV.

provider "google" {
  project = "my-project"
  # why this is not the default, i do not know
  billing_project = "my-project"
  user_project_override = true
  # ...
}

provider "google-beta" { ... } # same

# Make sure the relevant API is enabled
resource "google_project_service" "cloudidentity" {
  service = "cloudidentity.googleapis.com"
  disable_on_destroy = false
}

variable "google_workspace_customer_id" {
  type        = string
  description = "Your Google Workspace/Cloud Identity Customer ID ($ gcloud organizations list, e.g. Cxxxxxxxx)"
}

variable "google_workspace_domain" {
  type        = string
  description = "Your Google Workspace domain (e.g., yourcompany.com)."
}

locals {
  cluster_name = "clustername"
  secrets_group_name = "gke-${local.cluster_name}-${var.env}-secrets-team"
}

# --- Anchor Group: gke-security-groups ---
resource "google_cloud_identity_group" "gke_security_groups_anchor" {
  provider = google-beta

  display_name = "GKE Security Groups Anchor"
  group_key {
    id = "gke-security-groups@${var.google_workspace_domain}"
  }
  parent               = "customers/${var.google_workspace_customer_id}"
  initial_group_config = "WITH_INITIAL_OWNER"

  # Required label for standard mail-enabled groups.
  labels = {
    "cloudidentity.googleapis.com/groups.discussion_forum" = ""
  }

  # NOTE: "Each group must have the View members permission for Group members."
  # NOTE: Terraform cannot set the "Who can view members" permission directly.
  # You MUST verify/set this manually in the Google Groups UI/Admin Console after creation.
}

resource "google_cloud_identity_group" "secrets_team" {
  provider = google-beta

  display_name = title(replace(local.secrets_group_name, "-", " ")) # e.g., "Secrets Team"
  group_key {
    id = "${local.secrets_group_name}@${var.google_workspace_domain}" # e.g., secrets-team@yourcompany.com
  }
  parent               = "customers/${var.google_workspace_customer_id}"
  initial_group_config = "WITH_INITIAL_OWNER"

  # Required label for standard mail-enabled groups.
  labels = {
    "cloudidentity.googleapis.com/groups.discussion_forum" = ""
  }

  # NOTE: "Each group must have the View members permission for Group members."
  # NOTE: Terraform cannot set the "Who can view members" permission directly.
  # You MUST verify/set this manually in the Google Groups UI/Admin Console after creation.
}

resource "google_cloud_identity_group_membership" "nest_secrets_team" {
  provider = google-beta
  # Add to the anchor group...
  group = google_cloud_identity_group.gke_security_groups_anchor.id
  preferred_member_key {
    id = google_cloud_identity_group.secrets_team.group_key[0].id
  }
  roles {
    name = "MEMBER"
  }
  depends_on = [
    google_cloud_identity_group.secrets_team,
  ]
}

For step 2, again with terraform:

resource "google_container_cluster" {

  ...

  # --- Google Groups for RBAC Configuration ---
  authenticator_groups_config {
    security_group = google_cloud_identity_group.gke_security_groups_anchor.group_key[0].id
  }
}

Footnotes

1. I don't know why your group must be named that, and I can't be bothered testing to see if it's actually true. ↩