Skip to content

Commit 5351af8

Browse files
bavarianbidibavarianbidi
authored
docs: document operator update path... (#216)
... and compatibility matrix --------- Signed-off-by: Mario Constanti <[email protected]>
1 parent 114eda2 commit 5351af8

File tree

6 files changed

+236
-15
lines changed

6 files changed

+236
-15
lines changed

‎README.md

+11-10
Original file line numberDiff line numberDiff line change
@@ -33,16 +33,17 @@
3333

3434
### Garm Version
3535

36-
garm-operator uses [`garm-api-client`](https://github.com/cloudbase/garm/tree/main/client) to talk
37-
with `garm` servers. The supported `garm` server version is determined by `garm-api-client`.
36+
`garm-operator` is tightly coupled with the `garm` server. Therefore, the compatibility between `garm-operator` and `garm` is crucial.
3837

39-
> [!WARNING]
40-
> `garm-operator` is currently tested with `garm` server version `v0.1.4`. Newer versions are not guaranteed to work.
41-
>
42-
> `garm` server version `>v0.1.4` introduces a few breaking changes.
43-
> [provider]() and [credential]() configuration has moved away from being defined in the `garm` server configuration.
44-
> Instead, they are now defined via the `garm` API itself.
45-
> This feature set isn't implemented in the `garm-operator` yet.
38+
| garm-operator | garm | compatible |
39+
|---------------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
40+
| `0.4.x` | `0.1.5` | :white_check_mark: |
41+
| `0.4.x` | `0.1.4` | :x: |
42+
| `0.3.x` | `0.1.4` | :white_check_mark: |
43+
| `0.3.x` | `0.1.5` | :warning: It is not recommended to operate during a longer time period. This state mostly exists when you update the `garm-operator` from `0.3.x` to `0.4.x` |
44+
45+
> [!NOTE]
46+
> please also read the [operator update guide](docs/operator_update.md) when you want to update the `garm-operator`.
4647
4748
### Kubernetes Version
4849

@@ -92,7 +93,7 @@ For local development, please read the [development guide](DEVELOPMENT.md).
9293

9394
To make some assumptions and corresponding decisions transparent, we use ADRs (Architecture Decision Records) to document them.
9495

95-
All ADRs can be found in the [here](docs/architectural-decision-records.md).
96+
All ADRs can be found [here](docs/architectural-decision-records.md).
9697

9798
## Contributing
9899

‎docs/adrs/github_default_endpoint.md

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
<!-- SPDX-License-Identifier: MIT -->
2+
3+
---
4+
date: 2024-11-20
5+
desc: Reflection of the default GitHub endpoint
6+
state: accepted
7+
---
8+
<!--
9+
What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.?
10+
-->
11+
12+
<!--
13+
This is a basic ADR template from [Documenting architecture decisions - Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions).
14+
15+
It's possible to manage the ADR files with [adr-tools](https://github.com/npryce/adr-tools).
16+
-->
17+
# Reflection of the default GitHub endpoint
18+
19+
## Context and Problem Statement
20+
21+
With `garm` in version `0.1.5`, the GitHub API endpoints aren't defined in the configuration file anymore. The GitHub API endpoints are now defined via the `garm` API.
22+
23+
`garm` also ships with a default immutable GitHub API endpoint and points to the public GitHub API.
24+
25+
With the new API version `v1beta1` for all `garm-operator` related resources, we have to make use of the new `GitHubEndpoint` object to define the connection to the GitHub API.
26+
The `GitHubEndpoint` is referenced in the `GitHubCredential` object.
27+
28+
## Decision Drivers
29+
30+
* The default GitHub API endpoint is immutable.
31+
* The GitHub API endpoint must be configured via a new `GitHubEndpoint` object which then got referenced in the `GitHubCredential` object.
32+
33+
## Considered Options
34+
35+
* Reflecting the default GitHub API endpoint back into the kubernetes cluster by creating a `GitHubEndpoint` object with the default values.
36+
But as the default GitHub API endpoint is immutable, we have to catch this case in the operator and forbid any update/delete operations on this reflected `GitHubEndpoint`.
37+
38+
## Pros and Cons of the Options
39+
40+
#### Pros:
41+
* The code must not handle the immutability of the default GitHub API endpoint.
42+
43+
#### Cons:
44+
* Everyone who wants to use the `garm-operator` with the default GitHub API endpoint has to create a `GitHubEndpoint` object.
45+
46+
## Decision Outcome
47+
48+
The validation rules in the `CRD` blocks the creation of a `GitHubEndpoint` object with the name `github.com`.

‎docs/architectural-decision-records.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
In this file we are collecting all our DHC CaaS architectural decision records.
66

7-
| Date | Description |
8-
|:-----------|:----------------------------------------------------|
9-
| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) |
7+
| Date | Description |
8+
|:-----------|:-----------------------------------------------------------------------------|
9+
| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) |
10+
| 2023-12-04 | [Garm Server Initialization](adrs/garm_server_initialization.md) |
11+
| 2024-11-20 | [Reflection of the default GitHub endpoint](adrs/github_default_endpoint.md) |

‎docs/kube-state-metrics/kube-state-metrics-config.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
<!-- SPDX-License-Identifier: MIT -->
22

3-
# Kube State Metrics Configuration
3+
# kube-state-metrics Configuration
44

5-
[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for a kube-state-metrics agent to expose metrics of `garm-operators` custom resources.
5+
[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics) to expose metrics of `garm-operators` custom resources.
66
If you are using the official [helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) you can place the contents of `.data.config.yaml` into your helm-charts `values.yaml` file like so:
77

88
```yaml

‎docs/operator_update.md

+161
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
<!-- SPDX-License-Identifier: MIT -->
2+
3+
# compatibility
4+
5+
<!-- toc -->
6+
- [update to operator 0.4.x](#update-to-operator-04x)
7+
- [1. update <code>garm</code> to version <code>&gt;=0.1.5</code>](#1-update-garm-to-version-015)
8+
- [2. update <code>garm-operator</code> to version <code>0.4.x</code>](#2-update-garm-operator-to-version-04x)
9+
- [3. create new <code>CustomResources</code>](#3-create-new-customresources)
10+
<!-- /toc -->
11+
12+
## update to operator 0.4.x
13+
14+
> [!WARNING]
15+
> `garm-operator` in version `0.4.x` is not compatible with `garm` in version `0.1.4`.
16+
> If you connect `garm-operator` in version `0.4.x` to `garm` in version `0.1.4`,
17+
> the operator will stop working as we check the `garm` version in the operator.
18+
19+
### 1. update `garm` to version `>=0.1.5`
20+
21+
`garm-operator` in version `0.3.x` is compatible with `garm` in version `0.1.5`, if the `garm` server got updated from `0.1.4` to `0.1.5`.
22+
This is because `garm` is doing some migration steps by moving parts of the configuration from the `garm` server to the `garm` API.
23+
These newly introduced API endpoints are not by `garm-operator` in version `0.3.x`.
24+
25+
Therefore it's possible to still work with the existing `CustomResources`.
26+
27+
### 2. update `garm-operator` to version `0.4.x`
28+
29+
Once `garm` got updated to version `0.1.5`, you can update the `garm-operator` to version `0.4.x`.
30+
31+
### 3. create new `CustomResources`
32+
33+
As `garm` moved some configuration parts to the API, you have to create a few new `CustomResources`.
34+
Without these new `CustomResources`, `garm-operator` is not able to reconcile `Enterprises`, `Organizations` and `Repository` objects.
35+
36+
#### 3.1 create a `GarmServerConfig` object
37+
38+
It's now possible to define the `callbackUrl`, `metadataUrl` and `webhookUrl` via a `GarmServerConfig` object. Therefore it's not needed anymore to restart the `garm` server to apply these changes.
39+
40+
```yaml
41+
apiVersion: garm-operator.mercedes-benz.com/v1beta1
42+
kind: GarmServerConfig
43+
metadata:
44+
name: garm-server-config
45+
namespace: garm-operator-system
46+
spec:
47+
callbackUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/callbacks
48+
metadataUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/metadata
49+
webhookUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/webhook
50+
```
51+
52+
By running `kubectl get garmserverconfig` you should see the newly created `GarmServerConfig` object.
53+
54+
```bash
55+
NAME ID VERSION AGE
56+
garm-server-config dd2524b9-0789-499d-ba7d-3dba65cf9d3f v0.1.5 16h
57+
```
58+
59+
More information about the defined Urls can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#controller-operations).
60+
61+
#### 3.2 create `GitHubEndpoint` objects
62+
63+
The `GitHubEndpoint` objects are used to define the connection to the GitHub API. It's possible to define multiple `GitHubEndpoint` objects to connect to different GitHub instances.
64+
65+
```yaml
66+
apiVersion: garm-operator.mercedes-benz.com/v1beta1
67+
kind: GitHubEndpoint
68+
metadata:
69+
name: github-enterprise
70+
namespace: garm-operator-system
71+
spec:
72+
description: "github"
73+
apiBaseUrl: "https://github.com"
74+
uploadBaseUrl: "https://uploads.github.com"
75+
baseUrl: "https://api.github.com/"
76+
```
77+
78+
By running `kubectl get githubendpoint` you should see the newly created `GitHubEndpoint` object.
79+
80+
```bash
81+
NAME URL READY AGE
82+
github https://github.com True 3m2s
83+
```
84+
85+
More information about the github endpoints can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-endpoints)
86+
87+
> [!NOTE]
88+
> `garm` already ships a default `Github` object. But as this object is immutable, and we do not wanted to reflect the object into the `garm-operator`, we decided to create a new `CustomResource` for this. See the [Reflection of the default GitHub endpoint ADR](docs/adrs/github_default_endpoint.md) for more information.
89+
90+
#### 3.2 create `GitHubCredential` objects
91+
92+
With the new `v1beta1` of `Enterprises`, `Organizations` or `Repositories`, the reference to the Github credential has changed.
93+
In the previous `v1alpha1`, the used credential was referenced by the `.spec.credentialsName` field, which then pointed to a credential object, specified in the `garm` server configuration.
94+
95+
With the new `v1beta1`, the reference to the credential is done by the `.spec.credentialRef` field, which points to a `GitHubCredential` object.
96+
97+
> [!NOTE]
98+
> As long as `GitHubCredential` is not applied in `v1beta1`, the `garm-operator` will quit reconciling `Enterprises`, `Organizations` or `Repositories`.
99+
> The current assumption is, that a `GitHubCredential`, which was set in `.spec.credentialsName` in `v1alpha1`, will be created as `GitHubCredential` in `v1beta1` with the same name.
100+
> If this is not the case, the `garm-operator` will not be able to reconcile the `Enterprises`, `Organizations` or `Repositories` objects.
101+
> This state can be seen in the `status.conditions` field of these objects.
102+
>
103+
> ```yaml
104+
> status:
105+
> conditions:
106+
> - lastTransitionTime: "2024-11-14T20:01:27Z"
107+
> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found
108+
> reason: CredentialsRefFailed
109+
> status: "False"
110+
> type: Ready
111+
> - lastTransitionTime: "2024-11-14T20:01:27Z"
112+
> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found
113+
> reason: CredentialsRefFailed
114+
> status: "False"
115+
> type: CredentialsRef
116+
> [...]
117+
> ```
118+
119+
Create the `GitHubCredential` object and a corresponding `Secret` object with the PAT token.
120+
121+
```yaml
122+
apiVersion: garm-operator.mercedes-benz.com/v1beta1
123+
kind: GitHubCredential
124+
metadata:
125+
name: github-pat
126+
namespace: garm-operator-system
127+
spec:
128+
description: PAT for github
129+
endpointRef:
130+
apiGroup: garm-operator.mercedes-benz.com
131+
kind: GitHubEndpoint
132+
name: github
133+
authType: pat
134+
secretRef:
135+
name: github-pat
136+
key: token
137+
---
138+
apiVersion: v1
139+
kind: Secret
140+
metadata:
141+
name: github-pat
142+
namespace: garm-operator-system
143+
data:
144+
token: <base64 encoded PAT>
145+
```
146+
147+
More information about the github credentials can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-credentials)
148+
149+
By running `kubectl get githubcredential` you should see the newly created `GitHubCredential` object.
150+
151+
```bash
152+
NAME ID READY AUTHTYPE GITHUBENDPOINT AGE
153+
github-pat 1 True pat github 3m51s
154+
```
155+
156+
By running `kubectl get githubcredential -o wide` you see all the `Enterprise`, `Organization` or `Repository` objects which are using this `GitHubCredential`.
157+
158+
```bash
159+
NAME ID READY ERROR AUTHTYPE GITHUBENDPOINT REPOSITORIES ORGANIZATIONS ENTERPRISES AGE
160+
github-pat 1 True pat github ["my-org"] 4m46s
161+
```

‎docs/readme.md

+9
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,20 @@
44

55
<!-- toc -->
66
- [Quickstart](#quickstart)
7+
- [<a href="architectural-decision-records.md">Architecture Decision Records</a>](#architecture-decision-records)
78
- [how to](#how-to)
89
- [scale runners](#scale-runners)
910
- [pause reconciliation](#pause-reconciliation)
11+
- [<a href="config/configuration-parsing.md">configure the operator</a>](#configure-the-operator)
12+
- [<a href="kube-state-metrics/kube-state-metrics-config.md">monitor operator CRs</a>](#monitor-operator-crs)
1013
<!-- /toc -->
1114

1215
## Quickstart
1316

1417
[Quickstart](quickstart.md) contains a walk-through of how to get started with `garm-operator`.
1518

19+
## [Architecture Decision Records](architectural-decision-records.md)
20+
1621
## how to
1722

1823
### scale runners
@@ -97,3 +102,7 @@ $ kubectl annotate organization developer-org garm-operator.mercedes-benz.com/pa
97102

98103
organization.garm-operator.mercedes-benz.com/developer-org annotate
99104
```
105+
106+
### [configure the operator](config/configuration-parsing.md)
107+
108+
### [monitor operator CRs](kube-state-metrics/kube-state-metrics-config.md)

0 commit comments

Comments
 (0)