Skip to content

Commit a1ffecf

Browse files
lindehdp617
linde
and
hdp617
authored
feat: add an attached cluster example for kind as well (#677)
* added a kind example with an oidc module as well * added more info on credentials * Update main.tf just noticed a content-type typo * Update README.md just updated some text in the example * Update README.md missed an env var update for PRINCIPAL * Update README.md and another env var fixup * skipping commit from contributor's wrong account * putting back some license and typo fixes * Update .gitignore --------- Co-authored-by: Huy Pham <[email protected]>
1 parent fcaf9cb commit a1ffecf

File tree

8 files changed

+380
-0
lines changed

8 files changed

+380
-0
lines changed

‎anthos-attached-clusters/kind/README.md

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Sandbox Example to Attach a [kind](https://kind.sigs.k8s.io/) Cluster using Terraform
2+
3+
## Prerequisites
4+
The sample is meant just to provide a local example for experimentation. It assumes an environment where [`kind`](https://kind.sigs.k8s.io/) is available and could otherwise be run on the command line, e.g. `kind create cluster`.
5+
6+
## A note on providers
7+
8+
The other examples and module limit dependancies to terraform core providers, but this example takes advantage of some community supplied [providers](provider.tf). They're widely used for their purpose, but please review and consider.
9+
10+
## Usage
11+
12+
1. Edit the values in the terraform.tfvars file to suit your needs. Descriptions for each variable
13+
can be found in `variables.tf`. Additional optional features are also available and commented out
14+
in the `google_container_attached_cluster` resource in `main.tf`.
15+
16+
If you modify the cluster creation, ensure it meets
17+
[Cluster Prerequisites](https://cloud.google.com/anthos/clusters/docs/multi-cloud/attached/eks/reference/cluster-prerequisites).
18+
1. Initialize Terraform:
19+
```bash
20+
terraform init
21+
```
22+
1. Create and apply the plan:
23+
```bash
24+
terraform apply
25+
```
26+
Terraform may give a warning along the lines of `Warning: Content-Type is not recognized as a text type, got "application/jwk-set+json"` but this is ok and just a side effect of the `http` provider we're using and the content type the cluster returns for the `jwks` content.
27+
1. The process should take about a few short minutes to complete.
28+
1. Set some variables based on the terraform porjects values and use them to generate RBAC for the cluster and credentials to login:
29+
```bash
30+
31+
PROJECT=$(echo google_container_attached_cluster.primary.project | terraform console | tr -d '"')
32+
CLUSTER=$(echo google_container_attached_cluster.primary.name | terraform console | tr -d '"')
33+
KUBECONFIG=$(echo kind_cluster.cluster.kubeconfig_path | terraform console | tr -d '"')
34+
35+
# set this to whomever you'd like to grant access
36+
[email protected]
37+
# set this whatever role you intend
38+
ROLE=clusterrole/cluster-admin
39+
40+
gcloud container fleet memberships generate-gateway-rbac --apply \
41+
--kubeconfig ${KUBECONFIG} --context=kind-${CLUSTER} \
42+
--project=${PROJECT} \
43+
--membership=${CLUSTER} \
44+
--role=${ROLE} \
45+
--users=${PRINCIPAL}
46+
47+
gcloud container fleet memberships get-credentials ${CLUSTER} --project ${PROJECT}
48+
49+
kubectl get ns
50+
51+
```
52+
This will allow you to access the cluster using kubectl as you would other GKE Enterprise clusters, regardless of location (ie in GCP, other clouds, or on prem).
53+
54+
55+

‎anthos-attached-clusters/kind/main.tf

+123
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
18+
// This is an example of how you might use the attached module with a local kind cluster
19+
20+
locals {
21+
cluster_name = "${var.name_prefix}-cluster"
22+
}
23+
24+
resource "kind_cluster" "cluster" {
25+
name = local.cluster_name
26+
node_image = var.kind_node_image
27+
28+
kubeconfig_path = "${path.root}/.tmp/kube/${local.cluster_name}"
29+
30+
wait_for_ready = true
31+
32+
kind_config {
33+
kind = "Cluster"
34+
api_version = "kind.x-k8s.io/v1alpha4"
35+
feature_gates = {
36+
KubeletInUserNamespace : "true"
37+
}
38+
}
39+
}
40+
41+
provider "helm" {
42+
alias = "bootstrap_installer"
43+
kubernetes {
44+
host = kind_cluster.cluster.endpoint
45+
client_certificate = kind_cluster.cluster.client_certificate
46+
client_key = kind_cluster.cluster.client_key
47+
cluster_ca_certificate = kind_cluster.cluster.cluster_ca_certificate
48+
}
49+
}
50+
51+
module "attached_install_manifest" {
52+
source = "../modules/attached-install-manifest"
53+
attached_cluster_name = local.cluster_name
54+
attached_cluster_fleet_project = data.google_project.project.project_id
55+
gcp_location = var.gcp_location
56+
platform_version = var.platform_version
57+
providers = {
58+
helm = helm.bootstrap_installer
59+
}
60+
depends_on = [
61+
kind_cluster.cluster
62+
]
63+
}
64+
65+
data "google_project" "project" {
66+
project_id = var.gcp_project_id
67+
}
68+
69+
70+
71+
module "oidc" {
72+
source = "./oidc"
73+
74+
endpoint = kind_cluster.cluster.endpoint
75+
cluster_ca_certificate = kind_cluster.cluster.cluster_ca_certificate
76+
client_certificate = kind_cluster.cluster.client_certificate
77+
client_key = kind_cluster.cluster.client_key
78+
}
79+
80+
81+
resource "google_container_attached_cluster" "primary" {
82+
name = local.cluster_name
83+
project = data.google_project.project.project_id
84+
location = var.gcp_location
85+
description = "Kind attached cluster example"
86+
distribution = "generic"
87+
platform_version = var.platform_version
88+
oidc_config {
89+
issuer_url = module.oidc.issuer
90+
jwks = module.oidc.jwks
91+
}
92+
fleet {
93+
project = "projects/${data.google_project.project.number}"
94+
}
95+
96+
# Optional:
97+
# logging_config {
98+
# component_config {
99+
# enable_components = ["SYSTEM_COMPONENTS", "WORKLOADS"]
100+
# }
101+
# }
102+
103+
# Optional:
104+
# monitoring_config {
105+
# managed_prometheus_config {
106+
# enabled = true
107+
# }
108+
# }
109+
110+
# Optional:
111+
# authorization {
112+
# admin_users = ["[email protected]", ]
113+
# admin_groups = ["[email protected]", "[email protected]"]
114+
# }
115+
116+
depends_on = [
117+
module.attached_install_manifest
118+
]
119+
}
120+
121+
122+
123+

‎anthos-attached-clusters/kind/oidc/main.tf

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
// this is a local library to manage getting OIDC info from the cluster
18+
19+
data "http" "issuer" {
20+
21+
provider = http-full
22+
23+
url = "${var.endpoint}/.well-known/openid-configuration"
24+
request_headers = {
25+
content-type = "application/json"
26+
}
27+
28+
ca = var.cluster_ca_certificate
29+
client_crt = var.client_certificate
30+
client_key = var.client_key
31+
}
32+
33+
locals {
34+
issuer_json = jsondecode(data.http.issuer.response_body)
35+
}
36+
37+
data "http" "jwks" {
38+
39+
provider = http-full
40+
41+
url = local.issuer_json.jwks_uri
42+
request_headers = {
43+
content-type = "application/json"
44+
}
45+
46+
ca = var.cluster_ca_certificate
47+
client_crt = var.client_certificate
48+
client_key = var.client_key
49+
}
50+

‎anthos-attached-clusters/kind/oidc/outputs.tf

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
output "issuer" {
18+
value = local.issuer_json.issuer
19+
}
20+
21+
output "jwks" {
22+
value = base64encode(data.http.jwks.response_body)
23+
}

‎anthos-attached-clusters/kind/oidc/providers.tf

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
terraform {
18+
required_providers {
19+
http-full = {
20+
source = "salrashid123/http-full"
21+
}
22+
}
23+
}
24+

‎anthos-attached-clusters/kind/oidc/variables.tf

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
variable "endpoint" {
18+
type = string
19+
}
20+
21+
22+
variable "cluster_ca_certificate" {
23+
type = string
24+
}
25+
26+
variable "client_certificate" {
27+
type = string
28+
}
29+
30+
variable "client_key" {
31+
type = string
32+
}
33+
34+

‎anthos-attached-clusters/kind/provider.tf

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
terraform {
18+
required_providers {
19+
kind = {
20+
source = "tehcyx/kind"
21+
}
22+
}
23+
}
24+

‎anthos-attached-clusters/kind/variables.tf

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
/**
2+
* Copyright 2024 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
variable "name_prefix" {
18+
description = "Common prefix to use for generating names"
19+
type = string
20+
}
21+
22+
variable "gcp_project_id" {
23+
description = "The GCP project id where the cluster will be registered"
24+
type = string
25+
}
26+
27+
variable "gcp_location" {
28+
description = "GCP location to create the attached resource in"
29+
type = string
30+
default = "us-west1"
31+
}
32+
33+
variable "platform_version" {
34+
description = "Platform version of the attached cluster resource"
35+
type = string
36+
default = "1.28.0-gke.3"
37+
}
38+
39+
40+
variable "kind_node_image" {
41+
description = "The image used for the kind cluster"
42+
type = string
43+
default = "kindest/node:v1.28.0"
44+
}
45+
46+
47+

0 commit comments

Comments
 (0)