Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4 #54653

ikehz · 2017-10-26T20:57:47Z

What this PR does / why we need it: Add metrics exporters to the metadata-proxy addon for GCE. Work toward #8867.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Release note:

NONE

ikehz · 2017-10-26T20:58:04Z

Block on GoogleCloudPlatform/k8s-metadata-proxy#5.

ikehz · 2017-10-26T21:02:07Z

cluster/addons/metadata-proxy/gce/metadata-proxy.yaml

@@ -40,13 +42,22 @@ spec:
          limits:
            memory: "32Mi"
            cpu: "50m"
+      # BEGIN_PROMETHEUS_TO_SD
+      - name: prometheus-to-sd-exporter
+        image: gcr.io/google_containers/prometheus-to-sd:v0.1.2


@loburm Should this be v0.2.2?

Yes, please use 0.2.2 (as I remember 0.1.2 even don't have --source flag). Also it may have sense to add pod_name and pod_namespace, see #54635

Just pay attention that in future we will change pod name to pod id (currently not all kubernetes downward api support it)

ikehz · 2017-10-26T21:20:55Z

/assign @crassirostris

cjwagner · 2017-10-26T22:39:54Z

/retest

cjwagner · 2017-10-26T23:33:54Z

/retest

loburm · 2017-10-27T08:02:57Z

cluster/addons/metadata-proxy/gce/metadata-proxy.yaml

+        image: gcr.io/google_containers/prometheus-to-sd:v0.2.2
+        command:
+          - /monitor
+          - --stackdriver-prefix={{ prometheus_to_sd_prefix }}/addons/metadata_proxy


you don't need to add metadata_proxy as a prefix. Metric name is built as:
prefix/component_name/(metric_name without component_name)

Done; how does it know the component_name?

It's taken from the first part of --source flag. So if you specify multiple sources, each one is going to have a separate stackdriver prefix.

loburm · 2017-10-27T08:05:18Z

cluster/gce/gci/configure-helper.sh

@@ -1864,6 +1864,8 @@ EOF
  fi
  if [[ "${ENABLE_METADATA_PROXY:-}" == "simple" ]]; then
    setup-addon-manifests "addons" "metadata-proxy/gce"
+    local -r metadata_proxy_yaml="${dst_dir}/metadata-proxy/gce/metadata-proxy.yaml"
+    update-prometheus-to-sd-parameters ${metadata_proxy_yaml}


The same is required for the corresponding file in container-linux dir.

crassirostris

SGTM, but would wait until @loburm lgtms

crassirostris · 2017-10-27T12:04:08Z

cluster/addons/metadata-proxy/gce/metadata-proxy.yaml

-      - name: config-volume
-        configMap:
-          name: metadata-proxy-config
+      - name: ssl-certs


Please remove this volume, this is not a requirement for prom-to-sd and will be removed across all of K8s soon

ikehz · 2017-10-30T17:59:33Z

@loburm ptal.

ikehz · 2017-10-30T18:00:40Z

(Tests aren't passing because v0.1.4 doesn't exist yet (xref GoogleCloudPlatform/k8s-metadata-proxy#5). Waiting on #52744.

loburm · 2017-10-31T07:10:43Z

/lgtm

ikehz · 2017-10-31T18:13:19Z

/assign @cjcullen

ikehz · 2017-10-31T18:20:13Z

https://github.com/GoogleCloudPlatform/k8s-metadata-proxy/releases/tag/v0.1.4 now exists.

ikehz · 2017-10-31T18:20:17Z

/retest

ikehz · 2017-10-31T23:46:53Z

Addressed @cjcullen's comments, plus another rebase.

ikehz · 2017-10-31T23:47:03Z

Gah, didn't mean to close.

cjcullen · 2017-10-31T23:51:28Z

/lgtm

ikehz · 2017-11-02T15:00:56Z

Added change in e2e image, which was causing e2e failures. Ptal.

loburm · 2017-11-02T17:22:38Z

/lgtm

…to v0.1.4 and e2e to v0.0.2; remove configmag

mikedanese · 2017-11-03T19:56:48Z

/lgtm
/approve

k8s-github-robot · 2017-11-03T19:57:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cjcullen, ihmccreery, loburm, mikedanese

Associated issue: 8867

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~cluster/addons/metadata-proxy/OWNERS~~ [cjcullen,mikedanese]
~~cluster/gce/OWNERS~~ [mikedanese]
~~test/e2e/auth/OWNERS~~ [mikedanese]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-11-03T19:59:30Z

/test all [submit-queue is verifying that this PR is safe to merge]

k8s-ci-robot · 2017-11-03T20:26:03Z

@ihmccreery: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-gce-device-plugin-gpu	`be8aaf9`	link	`/test pull-kubernetes-e2e-gce-device-plugin-gpu`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-github-robot · 2017-11-03T20:46:09Z

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

shyamjvs · 2017-11-14T12:29:30Z

This PR seems to cause a scalability regression - #55695.

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 26, 2017

ikehz mentioned this pull request Oct 26, 2017

Bump to v0.1.4 GoogleCloudPlatform/k8s-metadata-proxy#5

Merged

k8s-github-robot assigned Q-Lee and cjcullen Oct 26, 2017

ikehz commented Oct 26, 2017

View reviewed changes

ikehz mentioned this pull request Oct 26, 2017

Instrument metadata proxy GoogleCloudPlatform/k8s-metadata-proxy#4

Merged

k8s-ci-robot assigned crassirostris Oct 26, 2017

ikehz mentioned this pull request Oct 26, 2017

Bump metadata proxy to 0.1.4 #52802

Closed

ikehz force-pushed the metadata-proxy-prom-to-sd branch from 4b5316f to 7cbcf6d Compare October 26, 2017 22:33

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 26, 2017

loburm reviewed Oct 27, 2017

View reviewed changes

crassirostris reviewed Oct 27, 2017

View reviewed changes

k8s-ci-robot assigned loburm Oct 31, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2017

ikehz force-pushed the metadata-proxy-prom-to-sd branch from c44a42d to b604133 Compare October 31, 2017 18:21

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2017

ikehz closed this Oct 31, 2017

ikehz reopened this Oct 31, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2017

ikehz force-pushed the metadata-proxy-prom-to-sd branch from 94be8fe to ce62faa Compare November 2, 2017 13:40

k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 2, 2017

ikehz force-pushed the metadata-proxy-prom-to-sd branch from 9e7980c to fa4493d Compare November 2, 2017 14:59

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 2, 2017

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 3, 2017

ikehz force-pushed the metadata-proxy-prom-to-sd branch from fa4493d to 1863b16 Compare November 3, 2017 17:17

k8s-github-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 3, 2017

Add prometheus-to-sd-exporter to metadata-proxy addon; bump to proxy …

be8aaf9

…to v0.1.4 and e2e to v0.0.2; remove configmag

ikehz force-pushed the metadata-proxy-prom-to-sd branch from 1863b16 to be8aaf9 Compare November 3, 2017 17:23

k8s-ci-robot assigned mikedanese Nov 3, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 3, 2017

k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 3, 2017

k8s-github-robot merged commit 2a40c48 into kubernetes:master Nov 3, 2017

shyamjvs mentioned this pull request Nov 14, 2017

metadata-proxy not able to handle too many pods/node #55695

Closed

shyamjvs mentioned this pull request Nov 14, 2017

Revert "Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4" #55710

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4 #54653

Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4 #54653

ikehz commented Oct 26, 2017

ikehz commented Oct 26, 2017

ikehz Oct 26, 2017

loburm Oct 27, 2017

ikehz commented Oct 26, 2017

cjwagner commented Oct 26, 2017

cjwagner commented Oct 26, 2017

loburm Oct 27, 2017

ikehz Oct 27, 2017

loburm Oct 31, 2017

loburm Oct 27, 2017

ikehz Oct 27, 2017

crassirostris left a comment

crassirostris Oct 27, 2017

ikehz Oct 27, 2017

ikehz commented Oct 30, 2017

ikehz commented Oct 30, 2017

loburm commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

cjcullen commented Oct 31, 2017

ikehz commented Nov 2, 2017

loburm commented Nov 2, 2017

mikedanese commented Nov 3, 2017

k8s-github-robot commented Nov 3, 2017

k8s-github-robot commented Nov 3, 2017

k8s-ci-robot commented Nov 3, 2017 •

edited

Loading

k8s-github-robot commented Nov 3, 2017

shyamjvs commented Nov 14, 2017

Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4 #54653

Add prometheus-to-sd-exporter to metadata-proxy addon; bump to v0.1.4 #54653

Conversation

ikehz commented Oct 26, 2017

ikehz commented Oct 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikehz commented Oct 26, 2017

cjwagner commented Oct 26, 2017

cjwagner commented Oct 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

crassirostris left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ikehz commented Oct 30, 2017

ikehz commented Oct 30, 2017

loburm commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

ikehz commented Oct 31, 2017

cjcullen commented Oct 31, 2017

ikehz commented Nov 2, 2017

loburm commented Nov 2, 2017

mikedanese commented Nov 3, 2017

k8s-github-robot commented Nov 3, 2017

k8s-github-robot commented Nov 3, 2017

k8s-ci-robot commented Nov 3, 2017 • edited Loading

k8s-github-robot commented Nov 3, 2017

shyamjvs commented Nov 14, 2017

k8s-ci-robot commented Nov 3, 2017 •

edited

Loading