Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Feat integrate spiffe #1663

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions cmd/nfd-master/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ func main() {
args.Overrides.ResyncPeriod = overrides.ResyncPeriod
case "nfd-api-parallelism":
args.Overrides.NfdApiParallelism = overrides.NfdApiParallelism
case "enable-spiffe":
args.Overrides.EnableSpiffe = overrides.EnableSpiffe
}
})

Expand Down Expand Up @@ -140,6 +142,8 @@ func initFlags(flagset *flag.FlagSet) (*master.Args, *master.ConfigOverrideArgs)
flagset.Var(overrides.ResyncPeriod, "resync-period", "Specify the NFD API controller resync period.")
overrides.NfdApiParallelism = flagset.Int("nfd-api-parallelism", 10, "Defines the maximum number of goroutines responsible of updating nodes. "+
"Can be used for the throttling mechanism.")
overrides.EnableSpiffe = flagset.Bool("enable-spiffe", false,
"Enables the Spiffe signature verification of created CRDs. This is still an EXPERIMENTAL feature.")

return args, overrides
}
4 changes: 4 additions & 0 deletions cmd/nfd-worker/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ func parseArgs(flags *flag.FlagSet, osArgs ...string) *worker.Args {
args.Overrides.LabelSources = overrides.LabelSources
case "no-owner-refs":
args.Overrides.NoOwnerRefs = overrides.NoOwnerRefs
case "enable-spiffe":
args.Overrides.EnableSpiffe = overrides.EnableSpiffe
}
})

Expand Down Expand Up @@ -131,6 +133,8 @@ func initFlags(flagset *flag.FlagSet) (*worker.Args, *worker.ConfigOverrideArgs)
flagset.Var(overrides.LabelSources, "label-sources",
"Comma separated list of label sources. Special value 'all' enables all sources. "+
"Prefix the source name with '-' to disable it.")
overrides.EnableSpiffe = flagset.Bool("enable-spiffe", false,
"Enables the Spiffe signature verification of created CRDs. This is still an EXPERIMENTAL feature.")

return args, overrides
}
6 changes: 6 additions & 0 deletions deployment/helm/node-feature-discovery/Chart.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dependencies:
- name: spire
repository: https://spiffe.github.io/helm-charts-hardened/
version: 0.24.1
digest: sha256:f3b4dc973a59682bf3aa5ca9b53322f57935dd093081e82a37b8082e00becbe9
generated: "2024-12-20T16:52:40.180416+01:00"
5 changes: 5 additions & 0 deletions deployment/helm/node-feature-discovery/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,8 @@ keywords:
- node-labels
type: application
version: 0.2.1
dependencies:
- name: spire
version: 0.24.1
repository: https://spiffe.github.io/helm-charts-hardened/
condition: spire.enabled
Binary file not shown.
14 changes: 14 additions & 0 deletions deployment/helm/node-feature-discovery/templates/master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -145,11 +145,25 @@ spec:
{{- with .Values.master.extraArgs }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.spire.enabled }}
- "-enable-spiffe"
{{- end }}
volumeMounts:
{{- if .Values.spire.enabled }}
- name: spire-agent-socket
mountPath: /run/spire/agent-sockets/api.sock
readOnly: true
{{- end }}
- name: nfd-master-conf
mountPath: "/etc/kubernetes/node-feature-discovery"
readOnly: true
volumes:
{{- if .Values.spire.enabled }}
- name: spire-agent-socket
hostPath:
path: /run/spire/agent-sockets/api.sock
type: Socket
Comment on lines +164 to +165
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I need come take back my earlier comment about the mounts 🤦‍♂️ If the spire-agent pod is deleted and re-created (for whatever reason) the nfd pods will permanently lose the spire sock (as the underlying socket changes), if I'm not mistaken. Thus a possible fix would be (and ditto nfd-worker).

            path: /run/spire/agent-sockets
            type: Directory

Thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. api.sock is a symlink to spire-agent.sock, so if spire-agent gets restarted or re-created we still have a symlink to the same file name that I think kubelet updates that periodically, right?

{{- end }}
- name: nfd-master-conf
configMap:
name: {{ include "node-feature-discovery.fullname" . }}-master-conf
Expand Down
14 changes: 14 additions & 0 deletions deployment/helm/node-feature-discovery/templates/worker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,18 @@ spec:
{{- with .Values.worker.extraArgs }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if .Values.spire.enabled }}
- "-enable-spiffe"
{{- end }}
ports:
- containerPort: {{ .Values.worker.port | default "8080"}}
name: http
volumeMounts:
{{- if .Values.spire.enabled }}
- name: spire-agent-socket
mountPath: /run/spire/agent-sockets/api.sock
readOnly: true
{{- end }}
- name: host-boot
mountPath: "/host-boot"
readOnly: true
Expand Down Expand Up @@ -144,6 +152,12 @@ spec:
mountPath: "/etc/kubernetes/node-feature-discovery"
readOnly: true
volumes:
{{- if .Values.spire.enabled }}
- name: spire-agent-socket
hostPath:
path: /run/spire/agent-sockets/api.sock
type: Socket
{{- end }}
- name: host-boot
hostPath:
path: "/boot"
Expand Down
58 changes: 56 additions & 2 deletions deployment/helm/node-feature-discovery/values.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
image:
repository: gcr.io/k8s-staging-nfd/node-feature-discovery
repository: docker.io/ahmedgrati/node-feature-discovery
# This should be set to 'IfNotPresent' for released version
pullPolicy: Always
# tag, if defined will use the given image tag, else Chart.AppVersion will be used
# tag
tag: v0.18.0-devel-105-gb1d33c2b2-dirty
imagePullSecrets: []

nameOverride: ""
Expand Down Expand Up @@ -574,3 +574,57 @@ prometheus:
enable: false
scrapeInterval: 10s
labels: {}

spire:
enabled: true
global:
spire:
clusterName: "nfd"
trustDomain: "nfd.k8s-sigs.io"
system:
name: "spire-system"
create: false
server:
name: "spire-server"
create: false
spire-agent:
nameOverride: "spire-agent"
kubeletConnectByHostname: "true"
server:
address: "nfd-spire-server.nfd"
workloadAttestors:
unix:
enabled: true
spire-server:
nameOverride: "spire-server"
controllerManager:
enabled: true
identities:
clusterStaticEntries:
node:
parentID: spiffe://nfd.k8s-sigs.io/spire/server
spiffeID: spiffe://nfd.k8s-sigs.io/root
selectors:
- k8s_psat:agent_ns:nfd
- k8s_psat:agent_sa:nfd-agent
- k8s_psat:cluster:nfd
nfd:
parentID: spiffe://nfd.k8s-sigs.io/root
spiffeID: spiffe://nfd.k8s-sigs.io/worker
selectors:
- k8s:pod-label:app.kubernetes.io/name:node-feature-discovery


caSubject:
commonName: "nfd.k8s-sigs.io"
country: "US"
organization: "SPIFFE"

upstream:
enabled: false
spiffe-csi-driver:
enabled: false
spiffe-oidc-discovery-provider:
enabled: false
tornjak-frontend:
enabled: false
16 changes: 16 additions & 0 deletions docs/reference/master-commandline-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,3 +306,19 @@ Example:
```bash
nfd-master -resync-period=2h
```

### -enable-spiffe

the `-enable-spiffe` flag enables SPIFFE verification for the created NodeFeature
objects created by the worker. When enabled, master verifies the signature that
is put on the annotations part of the NodeFeature object, and updates
Kubernetes nodes if the signature is verified. The feature should be enabled,
after deploying SPIFFE, and you can do it through the Helm chart.

Default: false.

Example:

```bash
nfd-master -enable-spiffe
```
16 changes: 16 additions & 0 deletions docs/reference/worker-commandline-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,3 +273,19 @@ Default: 0
Comma-separated list of `pattern=N` settings for file-filtered logging.

Default: *empty*

### -enable-spiffe

the `-enable-spiffe` flag enables signing NodeFeature spec on the worker side
and puts the signature in the annotations side of the NodeFeature object.
The signature is verified afterwards by the master. The feature
should be enabled, after deploying SPIFFE, and you can do it through
the Helm chart.

Default: false.

Example:

```bash
nfd-master -enable-spiffe
```
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ require (
github.com/prometheus/client_golang v1.21.0
github.com/smartystreets/goconvey v1.8.1
github.com/spf13/cobra v1.9.1
github.com/spiffe/go-spiffe/v2 v2.5.0
github.com/stretchr/testify v1.10.0
github.com/vektra/errors v0.0.0-20140903201135-c64d83aba85a
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56
Expand Down Expand Up @@ -71,6 +72,7 @@ require (
github.com/euank/go-kmsg-parser v2.0.0+incompatible // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/go-jose/go-jose/v4 v4.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
Expand Down Expand Up @@ -131,6 +133,7 @@ require (
github.com/stoewer/go-strcase v1.3.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.etcd.io/etcd/api/v3 v3.5.16 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.16 // indirect
go.etcd.io/etcd/client/v3 v3.5.16 // indirect
Expand Down
6 changes: 6 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=
github.com/go-jose/go-jose/v4 v4.0.4 h1:VsjPI33J0SB9vQM6PLmNjoHqMQNGPiZ0rHL7Ni7Q6/E=
github.com/go-jose/go-jose/v4 v4.0.4/go.mod h1:NKb5HO1EZccyMpiZNbdUw/14tiXNyUJh188dfnMCAfc=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
Expand Down Expand Up @@ -255,6 +257,8 @@ github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spiffe/go-spiffe/v2 v2.5.0 h1:N2I01KCUkv1FAjZXJMwh95KK1ZIQLYbPfhaxw8WS0hE=
github.com/spiffe/go-spiffe/v2 v2.5.0/go.mod h1:P+NxobPc6wXhVtINNtFjNWGBTreew1GBUCwT2wPmb7g=
github.com/stoewer/go-strcase v1.3.0 h1:g0eASXYtp+yvN9fK8sH94oCIk0fau9uV1/ZdJ0AVEzs=
github.com/stoewer/go-strcase v1.3.0/go.mod h1:fAH5hQ5pehh+j3nZfvwdk2RgEgQjAoM8wodgtPmh1xo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
Expand All @@ -279,6 +283,8 @@ github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510 h1:S2dVYn90KE98chq
github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/zeebo/errs v1.4.0 h1:XNdoD/RRMKP7HD0UhJnIzUy74ISdGGxURlYG8HSWSfM=
github.com/zeebo/errs v1.4.0/go.mod h1:sgbWHsvVuTPHcqJJGQ1WhI5KbWlHYz+2+2C/LSEtCw4=
go.etcd.io/bbolt v1.3.11 h1:yGEzV1wPz2yVCLsD8ZAiGHhHVlczyC9d1rP43/VCRJ0=
go.etcd.io/bbolt v1.3.11/go.mod h1:dksAq7YMXoljX0xu6VF5DMZGbhYYoLUalEiSySYAS4I=
go.etcd.io/etcd/api/v3 v3.5.16 h1:WvmyJVbjWqK4R1E+B12RRHz3bRGy9XVfh++MgbN+6n0=
Expand Down
Loading