-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Windows update helm chart #1176
base: main
Are you sure you want to change the base?
Changes from 14 commits
666e7a3
1658e7a
1851cdc
947d71b
9afffb5
5ac01e8
b44541f
375caba
245e073
da33f5c
682cb94
95e0131
27e5b8a
087bea9
e4d1f4d
c74e4bd
27000ab
6ac8f5a
59565e7
ed38405
cd9aee2
198c463
73cce96
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,7 +2,7 @@ | |
apiVersion: apps/v1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we remove this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In that scenario, e2e-resources when installed will always have 2 windows pods stuck in "Pending" because they're looking for a node. |
||
kind: Deployment | ||
metadata: | ||
name: windows-server-2019 | ||
name: {{ .Release.Name }}-windows-server-2019 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
labels: | ||
app: windows-server-2019 | ||
spec: | ||
|
@@ -17,18 +17,22 @@ spec: | |
spec: | ||
nodeSelector: | ||
kubernetes.io/os: windows | ||
cloud.google.com/gke-windows-os-version: '2019' | ||
node.kubernetes.io/windows-build: 10.0.17763 | ||
tolerations: | ||
- key: "os" | ||
operator: "Equal" | ||
value: "windows" | ||
effect: "NoSchedule" | ||
- key: "windows-build" | ||
operator: "Equal" | ||
value: "10.0.17763" | ||
effect: "NoSchedule" | ||
containers: | ||
- name: windows-server-2019 | ||
image: mcr.microsoft.com/windows/servercore:ltsc2019 | ||
resources: | ||
requests: | ||
cpu: "1" | ||
cpu: "1" | ||
memory: "300Mi" | ||
limits: | ||
cpu: "2" | ||
|
@@ -47,7 +51,7 @@ spec: | |
apiVersion: apps/v1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same thing here for Windows 2022 |
||
kind: Deployment | ||
metadata: | ||
name: windows-server-2022 | ||
name: {{ .Release.Name }}-windows-server-2022 | ||
labels: | ||
app: windows-server-2022 | ||
spec: | ||
|
@@ -62,12 +66,16 @@ spec: | |
spec: | ||
nodeSelector: | ||
kubernetes.io/os: windows | ||
cloud.google.com/gke-windows-os-version: '2022' | ||
node.kubernetes.io/windows-build: 10.0.20348 | ||
tolerations: | ||
- key: "os" | ||
operator: "Equal" | ||
value: "windows" | ||
effect: "NoSchedule" | ||
- key: "os" | ||
operator: "Equal" | ||
value: "windows" | ||
effect: "NoSchedule" | ||
- key: "windows-build" | ||
operator: "Equal" | ||
value: "10.0.20348" | ||
effect: "NoSchedule" | ||
containers: | ||
- name: windows-server-2022 | ||
image: mcr.microsoft.com/windows/servercore:ltsc2022 | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -136,6 +136,7 @@ integrations that you have configured. | |
| customSecretName | string | `""` | In case you don't want to have the license key in you values, this allows you to point to a user created secret to get the key from there. Can be configured also with `global.customSecretName` | | ||
| dnsConfig | object | `{}` | Sets pod's dnsConfig. Can be configured also with `global.dnsConfig` | | ||
| enableProcessMetrics | bool | `false` | Collect detailed metrics from processes running in the host. This defaults to true for accounts created before July 20, 2020. ref: https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes/new-relic-infrastructure-agent-1120 | | ||
| enableWindows | bool | `false` | Enables collection of metrics from Windows containers. Refer to the [Windows support](#tbd) section for more details. | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we have this link to a GH README while we wait for docs on NR? That way folks don't think it's a broken link. |
||
| fedramp.enabled | bool | `false` | Enables FedRAMP. Can be configured also with `global.fedramp.enabled` | | ||
| fullnameOverride | string | `""` | Override the full name of the release | | ||
| hostNetwork | bool | `false` | Sets pod's hostNetwork. Can be configured also with `global.hostNetwork` | | ||
|
@@ -192,6 +193,20 @@ integrations that you have configured. | |
| tolerations | list | `[]` | Sets pod's tolerations to node taints almost globally. (See [Affinities and tolerations](README.md#affinities-and-tolerations)) | | ||
| updateStrategy | object | See `values.yaml` | Update strategy for the deployed DaemonSets. | | ||
| verboseLog | bool | `false` | Sets the debug logs to this integration or all integrations if it is set globally. Can be configured also with `global.verboseLog` | | ||
| windowsNodeSelector | object | `{ kubernetes.io/os: windows, node.kubernetes.io/windows-build: BUILD_NUMBER }` | Sets windows pod's selector. Refer to [Windows support](#tbd) | | ||
| windowsOsList.agentImage | string | `""` | Overrides the infrastructure-agent windows image | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are we not able to auto-detect the image we want? We seem to already be using taints on a windows-flavoured deployment/daemonset. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can. This option is if people wanted to override the auto-selected images (like I am for development purposes) |
||
| windowsOsList.integrationImage | string | `""` | Overrides the nri-kubernetes windows image | | ||
|
||
## Running tests locally | ||
|
||
- Install Helm's 'chart-testing' utilities | ||
- `brew install chart-testing` | ||
- `brew install yamllint` | ||
- `helm plugin install https://github.com/helm-unittest/helm-unittest` | ||
- Run linter and yaml validation | ||
- `ct lint-and-install` from this folder | ||
- Run unit tests: | ||
- `helm unittest .` | ||
|
||
## Maintainers | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -33,8 +33,8 @@ We truncate at 63 chars because some Kubernetes name fields are limited to this | |
|
||
{{- /* Return a YAML with the mode to be added to the labels */ -}} | ||
{{- define "nriKubernetes._mode" -}} | ||
{{- if include "newrelic.common.privileged" . -}} | ||
mode: privileged | ||
{{- if and (include "newrelic.common.privileged" .) (not .Values.enableWindows) -}} | ||
mode: privileged | ||
{{- else -}} | ||
mode: unprivileged | ||
{{- end -}} | ||
|
@@ -116,3 +116,12 @@ readOnlyRootFilesystem: true | |
|
||
{{- toYaml $finalSecurityContext -}} | ||
{{- end -}} | ||
|
||
{{- define "windowsIntegrationImage" -}} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this the infrastructure image? If so, we may want to call it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We also may want to use namespacing of these helpers: |
||
{{ include "newrelic.common.images.image" ( dict "imageRoot" $.Values.images.integration "context" $ "imageTagSuffix" .imageTagSuffix) }} | ||
{{- end}} | ||
|
||
|
||
{{- define "windowsAgentImage" -}} | ||
{{ include "newrelic.common.images.image" ( dict "imageRoot" $.Values.images.agent "context" $ "imageTagSuffix" .imageTagSuffix) }} | ||
{{- end}} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,234 @@ | ||
{{- if and (.Values.kubelet.enabled) (.Values.enableWindows) }} | ||
{{- range .Values.windowsOsList }} | ||
apiVersion: apps/v1 | ||
kind: DaemonSet | ||
metadata: | ||
namespace: {{ $.Release.Namespace }} | ||
labels: | ||
kubernetes.io/os: windows | ||
{{- include "nriKubernetes.labels" $ | nindent 4 }} | ||
name: {{ join "-" (list (include "nriKubernetes.kubelet.fullname" $) .imageTagSuffix) }} | ||
{{- $legacyAnnotation:= fromYaml (include "newrelic.compatibility.annotations" $) -}} | ||
{{- with include "newrelic.compatibility.valueWithFallback" (dict "legacy" $legacyAnnotation "supported" $.Values.kubelet.annotations )}} | ||
annotations: {{ . | nindent 4 }} | ||
{{- end }} | ||
spec: | ||
{{- with $.Values.updateStrategy }} | ||
updateStrategy: {{ toYaml . | nindent 4 }} | ||
{{- end }} | ||
selector: | ||
matchLabels: | ||
{{- include "newrelic.common.labels.selectorLabels" $ | nindent 6 }} | ||
app.kubernetes.io/component: kubelet | ||
template: | ||
metadata: | ||
annotations: | ||
checksum/nri-kubernetes: {{ include (print $.Template.BasePath "/kubelet/scraper-configmap.yaml") $ | sha256sum }} | ||
checksum/agent-config: {{ include (print $.Template.BasePath "/kubelet/agent-configmap.yaml") $ | sha256sum }} | ||
{{- if include "newrelic.common.license.secret" $ }}{{- /* If the is secret to template */}} | ||
checksum/license-secret: {{ include (print $.Template.BasePath "/secret.yaml") $ | sha256sum }} | ||
{{- end }} | ||
checksum/integrations_config: {{ include (print $.Template.BasePath "/kubelet/integrations-configmap.yaml") $ | sha256sum }} | ||
{{- with $.Values.podAnnotations }} | ||
{{- toYaml . | nindent 8 }} | ||
{{- end }} | ||
labels: | ||
{{- include "nriKubernetes.labels.podLabels" $ | nindent 8 }} | ||
app.kubernetes.io/component: kubelet | ||
spec: | ||
{{- with include "newrelic.common.images.renderPullSecrets" ( dict "pullSecrets" (list $.Values.images.pullSecrets) "context" $) }} | ||
imagePullSecrets: | ||
{{- . | nindent 8 }} | ||
{{- end }} | ||
{{- with include "newrelic.common.dnsConfig" $ }} | ||
dnsConfig: | ||
{{- . | nindent 8 }} | ||
{{- end }} | ||
{{- with include "newrelic.common.priorityClassName" $ }} | ||
priorityClassName: {{ $ }} | ||
{{- end }} | ||
serviceAccountName: {{ include "newrelic.common.serviceAccount.name" $ }} | ||
hostNetwork: {{ include "nriKubernetes.kubelet.hostNetwork.value" $ }} | ||
{{- if include "nriKubernetes.kubelet.hostNetwork" $ }} | ||
dnsPolicy: ClusterFirstWithHostNet | ||
{{- end }} | ||
|
||
{{- if $.Values.kubelet.initContainers }} | ||
initContainers: {{- tpl ($.Values.kubelet.initContainers | toYaml) $ | nindent 8 }} | ||
{{- end }} | ||
containers: | ||
- name: kubelet | ||
image: {{ .integrationImage | default (include "windowsIntegrationImage" $) }} | ||
imagePullPolicy: {{ $.Values.images.integration.pullPolicy }} | ||
securityContext: | ||
windowsOptions: | ||
runAsUserName: "ContainerUser" | ||
env: | ||
- name: "NRI_KUBERNETES_SINK_HTTP_PORT" | ||
value: {{ get (fromYaml (include "nriKubernetes.kubelet.agentConfig" $)) "http_server_port" | quote }} | ||
- name: "NRI_KUBERNETES_CLUSTERNAME" | ||
value: {{ include "newrelic.common.cluster" $ }} | ||
- name: "NRI_KUBERNETES_VERBOSE" | ||
value: {{ include "newrelic.common.verboseLog.valueAsBoolean" $ | quote }} | ||
|
||
- name: "NRI_KUBERNETES_NODENAME" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "spec.nodeName" | ||
# Required to connect to the kubelet | ||
- name: "NRI_KUBERNETES_NODEIP" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "status.hostIP" | ||
|
||
{{- with $.Values.kubelet.extraEnv }} | ||
{{- toYaml . | nindent 12 }} | ||
{{- end }} | ||
{{- with $.Values.kubelet.extraEnvFrom }} | ||
envFrom: {{ toYaml . | nindent 12 }} | ||
{{- end }} | ||
volumeMounts: | ||
- name: nri-kubernetes-config | ||
mountPath: /etc/newrelic-infra/nri-kubernetes.yml | ||
subPath: nri-kubernetes.yml | ||
{{- with $.Values.kubelet.extraVolumeMounts }} | ||
{{- toYaml . | nindent 12 }} | ||
{{- end }} | ||
{{- with $.Values.kubelet.resources }} | ||
resources: {{ toYaml . | nindent 12 }} | ||
{{- end }} | ||
- name: agent | ||
image: {{ .agentImage | default (include "windowsAgentImage" $)}} | ||
args: [ "newrelic-infra" ] | ||
imagePullPolicy: {{ $.Values.images.agent.pullPolicy }} | ||
securityContext: | ||
windowsOptions: | ||
runAsUserName: "ContainerAdministrator" | ||
ports: | ||
- containerPort: {{ get (fromYaml (include "nriKubernetes.kubelet.agentConfig" $)) "http_server_port" }} | ||
env: | ||
- name: NRIA_LICENSE_KEY | ||
valueFrom: | ||
secretKeyRef: | ||
name: {{ include "newrelic.common.license.secretName" $ }} | ||
key: {{ include "newrelic.common.license.secretKeyName" $ }} | ||
|
||
- name: "NRIA_OVERRIDE_HOSTNAME_SHORT" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "spec.nodeName" | ||
|
||
- name: "NRIA_OVERRIDE_HOSTNAME" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "spec.nodeName" | ||
|
||
# Override NRIA_OVERRIDE_HOST_ROOT to empty if unprivileged. This must be done as an env var as the | ||
# `k8s-events-forwarder` and `infrastructure-bundle` images ship this very same env var set to /host. | ||
# Currently windows only runs in unprivileged mode. | ||
- name: "NRIA_OVERRIDE_HOST_ROOT" | ||
value: "" | ||
|
||
- name: "NRI_KUBERNETES_NODE_NAME" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "spec.nodeName" | ||
|
||
{{- if $.Values.useNodeNameAsDisplayName }} | ||
- name: "NRIA_DISPLAY_NAME" | ||
{{- if $.Values.prefixDisplayNameWithCluster }} | ||
value: "{{ include "newrelic.common.cluster" $ }}:$(NRI_KUBERNETES_NODE_NAME)" | ||
{{- else }} | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "spec.nodeName" | ||
{{- end }} | ||
{{- end }} | ||
|
||
{{- /* Needed to populate clustername in integration metrics */}} | ||
- name: "CLUSTER_NAME" | ||
value: {{ include "newrelic.common.cluster" $ }} | ||
- name: "NRIA_PASSTHROUGH_ENVIRONMENT" | ||
value: "CLUSTER_NAME" | ||
|
||
{{- /* Needed for autodiscovery since hostNetwork=false */}} | ||
- name: "NRIA_HOST" | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: "v1" | ||
fieldPath: "status.hostIP" | ||
|
||
{{- with $.Values.kubelet.extraEnv }} | ||
{{- toYaml . | nindent 12 }} | ||
{{- end }} | ||
{{- with $.Values.kubelet.extraEnvFrom }} | ||
envFrom: {{ toYaml . | nindent 12 }} | ||
{{- end }} | ||
volumeMounts: | ||
- name: config | ||
mountPath: C:\etc\newrelic-infra.yml | ||
subPath: newrelic-infra.yml | ||
- name: nri-integrations-cfg-volume | ||
mountPath: C:\etc\newrelic-infra\integrations.d\ | ||
- name: agent-tmpfs-data | ||
mountPath: C:\var\db\nnewrelic-infra\data | ||
- name: agent-tmpfs-user-data | ||
mountPath: C:\var\db\newrelic-infra\user_data | ||
- name: agent-tmpfs-tmp | ||
mountPath: C:\tmp | ||
{{- with $.Values.kubelet.extraVolumeMounts }} | ||
{{- toYaml $ | nindent 12 }} | ||
{{- end }} | ||
{{- with $.Values.kubelet.resources }} | ||
resources: {{ toYaml . | nindent 12 }} | ||
{{- end }} | ||
volumes: | ||
- name: agent-tmpfs-data | ||
emptyDir: {} | ||
- name: agent-tmpfs-user-data | ||
emptyDir: {} | ||
- name: agent-tmpfs-tmp | ||
emptyDir: {} | ||
- name: nri-kubernetes-config | ||
configMap: | ||
name: {{ include "nriKubernetes.kubelet.fullname" $ }} | ||
items: | ||
- key: nri-kubernetes.yml | ||
path: nri-kubernetes.yml | ||
- name: config | ||
configMap: | ||
name: {{ include "nriKubernetes.kubelet.fullname.agent" $ }} | ||
items: | ||
- key: newrelic-infra.yml | ||
path: newrelic-infra.yml | ||
- name: nri-integrations-cfg-volume | ||
configMap: | ||
name: {{ include "nriKubernetes.kubelet.fullname.integrations" $ }} | ||
{{- with $.Values.kubelet.extraVolumes }} | ||
{{- toYaml . | nindent 8 }} | ||
{{- end }} | ||
{{- with include "nriKubernetes.kubelet.affinity" $ }} | ||
affinity: | ||
{{- . | nindent 8 }} | ||
{{- end }} | ||
{{- with include "nriKubernetes.kubelet.tolerations" $ }} | ||
tolerations: | ||
{{- . | nindent 8 }} | ||
{{- end }} | ||
nodeSelector: | ||
{{- if $.Values.kubelet.windowsNodeSelector }} | ||
{{- toYaml $.Values.kubelet.windowsNodeSelector | nindent 8 }} | ||
{{- else }} | ||
kubernetes.io/os: windows | ||
# Windows containers can only be deployed on hosts with the same Windows version | ||
node.kubernetes.io/windows-build: {{ .buildNumber }} | ||
{{- end }} | ||
--- | ||
{{- end }} | ||
{{- end }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,8 +4,10 @@ kind: PodSecurityPolicy | |
metadata: | ||
name: privileged-{{ include "newrelic.common.naming.fullname" . }} | ||
spec: | ||
{{- if not .Values.enableWindows }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a change to the linux experience if this is off? Enabling windows shouldn't cause a regression on the linux experience, if it does, we should create a new security policy just for Windows. |
||
allowedCapabilities: | ||
- '*' | ||
{{- end }} | ||
fsGroup: | ||
rule: RunAsAny | ||
privileged: true | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we can't change releases that have already been cut. You'll need to stick to adding any new changes in the
unreleased
section. Details here