Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No visible metrics when apiserver connection fails #30

Closed
zuzzas opened this issue Jul 5, 2019 · 1 comment
Closed

No visible metrics when apiserver connection fails #30

zuzzas opened this issue Jul 5, 2019 · 1 comment
Labels
enhancement New feature or request
Milestone

Comments

@zuzzas
Copy link
Contributor

zuzzas commented Jul 5, 2019

There is not indication in metrics generated by shell-operator, when connection to kube-apiserver is unstable or not working at all.

The log is full of such messages:
E0705 10:45:58.761826 1 reflector.go:125] github.com/flant/shell-operator/pkg/kube_events_manager/kube_events_manager.go:354: Failed to list *unstructured.Unstructured: Get https://192.168.0.1:443/api/v1/namespaces/kube-system/endpoints?fieldSelector=metadata.name%3Dkube-dns&limit=500&resourceVersion=0: dial tcp 192.168.0.1:443: i/o timeout

Unfortunately, those are not as useful as a proper Prometheus metric.

@diafour diafour added this to the 1.0.0 milestone Oct 9, 2019
@diafour diafour added the enhancement New feature or request label Jan 24, 2020
@diafour diafour modified the milestones: 1.0.0, 1.0.0-beta.10 Apr 3, 2020
@diafour diafour modified the milestones: 1.0.0-beta.10, 1.0.0-beta.11 Jun 23, 2020
@diafour
Copy link
Contributor

diafour commented Jun 23, 2020

  • klog messages are wrapped in json
  • There is a new metric shell_operator_kubernetes_client_request_result_total. If connection is lost, it has label code="<error>". See feat: more metrics #180.

@diafour diafour closed this as completed Jun 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants