Skip to content

init-secrets job #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5e672fc
add secret generation package
gaelgatelement Jan 30, 2025
1807b67
matrix-tools: add generate-secrets command
gaelgatelement Jan 30, 2025
0d4c577
init-secrets: init templates
gaelgatelement Jan 30, 2025
086b7bc
ci values files: handle init-secrets
gaelgatelement Jan 30, 2025
ea8b3ce
tests/manifests: handle init-secrets job
gaelgatelement Jan 30, 2025
d3d7014
init-secrets: various fixes to templates & values
gaelgatelement Jan 30, 2025
17dda34
tests/integration: let init-secrets generate macaroon
gaelgatelement Jan 30, 2025
139cc8c
init-secrets: add rbac.create
gaelgatelement Jan 31, 2025
064c169
Refactor secret generation to include labels and improve validation
gaelgatelement Jan 31, 2025
6254977
Refactor secret handling logic
gaelgatelement Jan 31, 2025
f8c5319
matrix-tools: ensure we can create empty secret
gaelgatelement Jan 31, 2025
04c70ef
init-secrets: gate job/role creation behind rbac.create
gaelgatelement Jan 31, 2025
63b0739
init-secrets: automount serviceAccount token and configure namespace
gaelgatelement Jan 31, 2025
746c5d4
Refactor secret handling in matrix-stack charts
gaelgatelement Jan 31, 2025
d6ecbe4
init-secret: remove hostAliases
gaelgatelement Jan 31, 2025
1477d50
remove secrets package from pytest
gaelgatelement Jan 31, 2025
4d7f26d
add newsfragments
gaelgatelement Jan 31, 2025
938cf43
ci: increase matrix tools runner
gaelgatelement Jan 31, 2025
e769be8
init-secrets: name the secret generated
gaelgatelement Jan 31, 2025
aab9490
golintci: update go modules
gaelgatelement Jan 31, 2025
629643d
job: use helm hooks
gaelgatelement Jan 31, 2025
f73776e
golangci fix
gaelgatelement Jan 31, 2025
a372f0b
update matrix-tools to 0.2.0
gaelgatelement Jan 31, 2025
a10c7a8
init-secrets: fix checkov errors
gaelgatelement Jan 31, 2025
1197239
matrix-tools: better logging
gaelgatelement Jan 31, 2025
c34f17d
init-secrets: fix permissions
gaelgatelement Jan 31, 2025
67bc793
init-secrets: only render if some secrets must be generated
gaelgatelement Jan 31, 2025
960e15a
manifests tests: handle shared_components with more details
gaelgatelement Jan 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/matrix-tools.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ permissions:

jobs:
tests:
runs-on: cpu-s
runs-on: memory-m
container:
image: ghcr.io/${{ github.repository }}/ci-runner
credentials:
Expand All @@ -38,7 +38,9 @@ jobs:
run: echo "/usr/local/go/bin" >> "${GITHUB_PATH}"

- name: Lint with golangci-lint
run: ls -l /usr/local && echo $PATH && cd matrix-tools && golangci-lint run ./...
run: |
cd matrix-tools
golangci-lint run ./... -v --show-stats --no-config --modules-download-mode readonly --timeout 5m

- name: Run tests
run: cd matrix-tools && go test -v ./...
Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/element-web-checkov-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
additional: {
"default_server_config": {
Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/element-web-minimal-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
#
# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial

initSecrets:
enabled: false

elementWeb:
ingress:
host: element.ess.localhost
Expand Down
20 changes: 20 additions & 0 deletions charts/matrix-stack/ci/init-secrets-checkov-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Copyright 2024 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial

initSecrets:
enabled: true
annotations:
checkov.io/skip1: CKV_K8S_11=We deliberately don't set CPU limits. Pod is BestEffort not Guaranteed
checkov.io/skip2: CKV_K8S_43=No digests
checkov.io/skip3: CKV2_K8S_6=No network policy yet
checkov.io/skip4: CKV_K8S_38=The job needs a service account

elementWeb:
enabled: false

synapse:
enabled: false

wellKnownDelegation:
enabled: false
15 changes: 15 additions & 0 deletions charts/matrix-stack/ci/init-secrets-minimal-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Copyright 2024 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial

initSecrets:
enabled: true

elementWeb:
enabled: false

synapse:
enabled: false

wellKnownDelegation:
enabled: false
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/nothing-enabled-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
#
# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
5 changes: 5 additions & 0 deletions charts/matrix-stack/ci/pytest-element-web-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

serverName: ess.localhost

matrixTools:
image:
pullPolicy: Never
digest: ""

Comment on lines +7 to +11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed in the EW integration test values?

elementWeb:
ingress:
host: element.{{ $.Values.serverName }}
Expand Down
3 changes: 0 additions & 3 deletions charts/matrix-stack/ci/pytest-synapse-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@ synapse:
registrationSharedSecret:
secret: "{{ $.Release.Name }}-synapse-secrets"
secretKey: registrationSharedSecret
macaroon:
secret: "{{ $.Release.Name }}-synapse-secrets"
secretKey: macaroon
signingKey:
secret: "{{ $.Release.Name }}-synapse-secrets"
secretKey: signingKey
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/synapse-minimal-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/synapse-worker-example-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/well-known-checkov-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/well-known-minimal-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/ci/well-known-synapse-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

serverName: ess.localhost

initSecrets:
enabled: false

elementWeb:
enabled: false

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial

## The matrix-tools iamge, used in multiple components
matrixTools:
{{ image(registry="ghcr.io", repository="element-hq/ess-helm/matrix-tools", tag="0.1.1") | indent(2) }}
{{ image(registry="ghcr.io", repository="element-hq/ess-helm/matrix-tools", tag="0.2.0") | indent(2) }}

## The server name of the Matrix Stack. This gets embedded in user IDs & room IDs
## It can not change after the initial deployment.
Expand Down
42 changes: 42 additions & 0 deletions charts/matrix-stack/source/init-secrets.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"$id": "file://init-secrets",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
},
"rbac": {
"type": "object",
"properties": {
"create": {
"type": "boolean"
}
}
},
"labels": {
"$ref": "file://common/labels.json"
},
"annotations": {
"$ref": "file://common/workloadAnnotations.json"
},
"containersSecurityContext": {
"$ref": "file://common/containersSecurityContext.json"
},
"nodeSelector": {
"$ref": "file://common/nodeSelector.json"
},
"podSecurityContext": {
"$ref": "file://common/podSecurityContext.json"
},
"resources": {
"$ref": "file://common/resources.json"
},
"serviceAccount": {
"$ref": "file://common/serviceAccount.json"
},
"tolerations": {
"$ref": "file://common/tolerations.json"
}
}
}
20 changes: 20 additions & 0 deletions charts/matrix-stack/source/init-secrets.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{#
Copyright 2024 New Vector Ltd

SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
#}

{% import 'sub_schema_values.yaml.j2' as sub_schema_values -%}
enabled: true

rbac:
create: true

{{- sub_schema_values.labels() -}}
{{- sub_schema_values.workloadAnnotations() -}}
{{- sub_schema_values.containersSecurityContext() -}}
{{- sub_schema_values.nodeSelector() -}}
{{- sub_schema_values.podSecurityContext(user_id='10010', group_id='10010') -}}
{{- sub_schema_values.resources(requests_memory='50Mi', requests_cpu='50m', limits_memory='200Mi') -}}
{{- sub_schema_values.serviceAccount() -}}
{{- sub_schema_values.tolerations() -}}
3 changes: 3 additions & 0 deletions charts/matrix-stack/source/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@
"topologySpreadConstraints": {
"$ref": "file://common/topologySpreadConstraints.json"
},
"initSecrets": {
"$ref": "file://init-secrets.json"
},
"elementWeb": {
"$ref": "file://element-web.json"
},
Expand Down
3 changes: 3 additions & 0 deletions charts/matrix-stack/source/values.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
{{ sub_schema_values.ess() }}

## Components
initSecrets:
{% macro initSecrestValues() %}{% include 'init-secrets.yaml.j2'%}{% endmacro %}
{{- initSecrestValues() | trim | indent(2) }}

elementWeb:
{% macro elementWebValues() %}{% include 'element-web.yaml.j2'%}{% endmacro %}
Expand Down
59 changes: 47 additions & 12 deletions charts/matrix-stack/templates/ess-library/_credentials.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,53 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
{{- with required "element-io.ess-library.check-credential missing context" .context -}}
{{- $secretPath := .secretPath -}}
{{- $secretProperty := .secretProperty -}}
{{- if and .secretProperty.value (or .secretProperty.secret .secretProperty.secretKey) -}}
{{- fail (printf "The secret %s must either have a value, or both secret & secretKey properties" $secretPath) -}}
{{- else if and .secretProperty.secret (not .secretProperty.secretKey) -}}
{{- fail (printf "The secret %s has a secret but no secretKey property" $secretPath) -}}
{{- else if and .secretProperty.secretKey (not .secretProperty.secret) -}}
{{- fail (printf "The secret %s has a secretKey but no secret property" $secretPath) -}}
{{- else if and .secretProperty.secret .secretProperty.secretKey -}}
{{- /* OK secret has a secret and a secretKey, do nothing */ -}}
{{- else if .secretProperty.value -}}
{{- /* OK secret has a value, do nothing */ -}}
{{- else -}}
{{- fail (printf "The secret %s is missing its secret/secretKey properties" $secretPath) -}}
{{- $initIfAbsent := .initIfAbsent | default false -}}
{{- if not $initIfAbsent -}}
{{- if and .secretProperty.value (or .secretProperty.secret .secretProperty.secretKey) -}}
{{- fail (printf "The secret %s must either have a value, or both secret & secretKey properties" $secretPath) -}}
{{- else if and .secretProperty.secret (not .secretProperty.secretKey) -}}
{{- fail (printf "The secret %s has a secret but no secretKey property" $secretPath) -}}
{{- else if and .secretProperty.secretKey (not .secretProperty.secret) -}}
{{- fail (printf "The secret %s has a secretKey but no secret property" $secretPath) -}}
{{- else if and .secretProperty.secret .secretProperty.secretKey -}}
{{- /* OK secret has a secret and a secretKey, do nothing */ -}}
{{- else if .secretProperty.value -}}
{{- /* OK secret has a value, do nothing */ -}}
{{- else -}}
{{- fail (printf "The secret %s is missing its secret/secretKey properties" $secretPath) -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end }}

{{- define "element-io.ess-library.init-secret-path" -}}
{{- $root := .root -}}
{{- with required "element-io.ess-library.init-secret-path" .context -}}
{{- $secretProperty := required "element-io.ess-library.init-secret-path context missing secretProperty" .secretProperty -}}
{{- $initSecretKey := required "element-io.ess-library.init-secret-path context missing initSecretKey" .initSecretKey -}}
{{- $defaultSecretName := required "element-io.ess-library.init-secret-path context missing defaultSecretName" .defaultSecretName -}}
{{- $defaultSecretKey := required "element-io.ess-library.init-secret-path context missing defaultSecretKey" .defaultSecretKey -}}
{{- if not $secretProperty -}}
{{- if $root.Values.initSecrets.enabled -}}
{{- printf "%s/%s" (printf "%s-generated" $root.Release.Name) $initSecretKey -}}
{{- end -}}
{{- else -}}
{{- include "element-io.ess-library.provided-secret-path" (dict "root" $root "context" (dict "secretProperty" $secretProperty "defaultSecretName" $defaultSecretName "defaultSecretKey" $defaultSecretKey)) -}}
{{- end -}}
{{- end -}}
{{- end -}}


{{- define "element-io.ess-library.provided-secret-path" -}}
{{- $root := .root -}}
{{- with required "element-io.ess-library.provided-secret-path missing context" .context -}}
{{- $secretProperty := required "element-io.ess-library.provided-secret-path context missing secretProperty" .secretProperty -}}
{{- $defaultSecretName := required "element-io.ess-library.provided-secret-path context missing defaultSecretName" .defaultSecretName -}}
{{- $defaultSecretKey := required "element-io.ess-library.provided-secret-path context missing defaultSecretKey" .defaultSecretKey -}}
{{- if $secretProperty.value -}}
{{- printf "%s/%s" $defaultSecretName $defaultSecretKey -}}
{{- else -}}
{{- printf "%s/%s" (tpl $secretProperty.secret $root) (tpl $secretProperty.secretKey $root) -}}
{{- end -}}
{{- end -}}
{{- end -}}
28 changes: 28 additions & 0 deletions charts/matrix-stack/templates/init-secrets/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{{- /*
Copyright 2024 New Vector Ltd

SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
*/ -}}

{{- define "element-io.init-secrets.labels" -}}
{{- $root := .root -}}
{{- with required "element-io.init-secrets.labels missing context" .context -}}
{{ include "element-io.ess-library.labels.common" (dict "root" $root "context" .labels) }}
app.kubernetes.io/component: matrix-tools
app.kubernetes.io/name: init-secrets
app.kubernetes.io/instance: {{ $root.Release.Name }}-init-secrets
app.kubernetes.io/version: {{ $root.Values.matrixTools.image.tag }}
{{- end }}
{{- end }}

{{- define "element-io.init-secrets.generated-secrets" -}}
{{- $root := .root -}}
{{- with $root.Values.synapse }}
{{- if .enabled -}}
{{- if not .macaroon }}
- {{ (printf "%s-generated" $root.Release.Name) }}:SYNAPSE_MACAROON:rand32
{{- end }}
{{- end }}
{{- end }}
{{- end }}

Loading
Loading