Skip to content

Commit 8c1ec1f

Browse files
committed
[chore] Adopt for the kubelet cpu utilization metrics deprecation
The following metrics emitted by kubeletstats receiver were deprecated: - k8s.node.cpu.utilization - k8s.pod.cpu.utilization - container.cpu.utilization See open-telemetry/opentelemetry-collector-contrib#25901 for more details. Those metrics are already excluded by default in the signalfx exporter. So if we don't do anything, user will see the following confusing warnings: ``` 2024-02-06T19:21:44.773Z warn metadata/generated_metrics.go:2894 [WARNING] `container.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric container.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.773Z warn metadata/generated_metrics.go:2897 [WARNING] `k8s.node.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric k8s.node.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.773Z warn metadata/generated_metrics.go:2900 [WARNING] `k8s.pod.cpu.utilization` should not be enabled: This metric will be disabled in a future release. Use metric k8s.pod.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2894 [WARNING] `container.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric container.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2897 [WARNING] `k8s.node.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric k8s.node.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2900 [WARNING] `k8s.pod.cpu.utilization` should not be enabled: This metric will be disabled in a future release. Use metric k8s.pod.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2894 [WARNING] `container.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric container.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2897 [WARNING] `k8s.node.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric k8s.node.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2900 [WARNING] `k8s.pod.cpu.utilization` should not be enabled: This metric will be disabled in a future release. Use metric k8s.pod.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2894 [WARNING] `container.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric container.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2897 [WARNING] `k8s.node.cpu.utilization` should not be enabled: WARNING: This metric will be disabled in a future release. Use metric k8s.node.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} 2024-02-06T19:21:44.774Z warn metadata/generated_metrics.go:2900 [WARNING] `k8s.pod.cpu.utilization` should not be enabled: This metric will be disabled in a future release. Use metric k8s.pod.cpu.usage instead. {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics"} ``` Potentially we couldn't just explicitly disable them in the default k8s configuration on the receiver side. The problem is that the exclusion on the signalfx exporter side potentially can be disabled by users. For those users, we actually want to show the warnings. So we need to add another converter until the metric is disabled on the kubelet receiver side by default. The converter looks at the signalfx exporter in user's config, and if metrics are not explicitly enabled on the exporter side, it disables them on the kubelet receiver side to supress the warnings. The converted required to copy `dpfilters` package from signal exporter. This will be removed along with the converter in a few releases.
1 parent c1ecc0a commit 8c1ec1f

26 files changed

+1700
-2
lines changed

go.mod

+2-2
Original file line numberDiff line numberDiff line change
@@ -319,7 +319,7 @@ require (
319319
github.com/go-sql-driver/mysql v1.7.1 // indirect
320320
github.com/go-stack/stack v1.8.1 // indirect
321321
github.com/go-test/deep v1.1.0 // indirect
322-
github.com/gobwas/glob v0.2.4-0.20181002190808-e7a84e9525fe // indirect
322+
github.com/gobwas/glob v0.2.4-0.20181002190808-e7a84e9525fe
323323
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
324324
github.com/gogo/googleapis v1.4.1 // indirect
325325
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
@@ -466,7 +466,7 @@ require (
466466
github.com/ryanuber/go-glob v1.0.0 // indirect
467467
github.com/scaleway/scaleway-sdk-go v1.0.0-beta.21 // indirect
468468
github.com/shirou/gopsutil/v3 v3.24.1 // indirect
469-
github.com/signalfx/com_signalfx_metrics_protobuf v0.0.3 // indirect
469+
github.com/signalfx/com_signalfx_metrics_protobuf v0.0.3
470470
github.com/signalfx/defaults v1.2.2-0.20180531161417-70562fe60657 // indirect
471471
github.com/signalfx/gohistogram v0.0.0-20160107210732-1ccfd2ff5083 // indirect
472472
github.com/signalfx/ingest-protocols v0.2.0 // indirect
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
// Copyright Splunk, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package configconverter
16+
17+
import (
18+
"context"
19+
"fmt"
20+
"regexp"
21+
22+
sfxpb "github.com/signalfx/com_signalfx_metrics_protobuf/model"
23+
"go.opentelemetry.io/collector/confmap"
24+
25+
"github.com/signalfx/splunk-otel-collector/internal/configconverter/dpfilters"
26+
)
27+
28+
// Metrics deprecated by kubeletstats receiver that need to be disabled if not explicitly included in signalfx exporter.
29+
const (
30+
k8sNodeCPUUtilization = "k8s.node.cpu.utilization"
31+
k8sPodCPUUtilization = "k8s.pod.cpu.utilization"
32+
containerCPUUtilization = "container.cpu.utilization"
33+
)
34+
35+
// signalfxExporterConfig is the configuration for the signalfx exporter that contains the metrics filters.
36+
type signalfxExporterConfig struct {
37+
ExcludeMetrics []dpfilters.MetricFilter `mapstructure:"exclude_metrics"`
38+
IncludeMetrics []dpfilters.MetricFilter `mapstructure:"include_metrics"`
39+
}
40+
41+
// DisableKubeletUtilizationMetrics is a MapConverter that disables the following deprecated metrics:
42+
// - `k8s.node.cpu.utilization`
43+
// - `k8s.pod.cpu.utilization`
44+
// - `container.cpu.utilization`
45+
// The converter disables the metrics at the receiver level to avoid showing users a warning message because
46+
// they are excluded in signalfx exporter by default.
47+
// We don't disable them in case if users explicitly include them in signalfx exporter.
48+
type DisableKubeletUtilizationMetrics struct{}
49+
50+
func (DisableKubeletUtilizationMetrics) Convert(_ context.Context, cfgMap *confmap.Conf) error {
51+
if cfgMap == nil {
52+
return fmt.Errorf("cannot DisableKubeletUtilizationMetrics on nil *confmap.Conf")
53+
}
54+
55+
receivers, err := cfgMap.Sub("receivers")
56+
if err != nil {
57+
return nil // Ignore invalid config. Rely on the config validation to catch this.
58+
}
59+
kubeletReceiverConfigs := map[string]map[string]any{}
60+
for receiverName, receiverCfg := range receivers.ToStringMap() {
61+
if regexp.MustCompile(`kubeletstats(/\w+)?`).MatchString(receiverName) {
62+
if v, ok := receiverCfg.(map[string]any); ok {
63+
kubeletReceiverConfigs[receiverName] = v
64+
}
65+
}
66+
}
67+
68+
exporters, err := cfgMap.Sub("exporters")
69+
if err != nil {
70+
return nil // Ignore invalid config. Rely on the config validation to catch this.
71+
}
72+
sfxExporterConfigs := map[string]map[string]any{}
73+
for exporterName, exporterCfg := range exporters.ToStringMap() {
74+
if regexp.MustCompile(`signalfx(/\w+)?`).MatchString(exporterName) {
75+
if v, ok := exporterCfg.(map[string]any); ok {
76+
sfxExporterConfigs[exporterName] = v
77+
}
78+
}
79+
}
80+
81+
// If there is no signalfx exporter or kubeletstats receiver, there is nothing to do.
82+
if len(kubeletReceiverConfigs) == 0 || len(sfxExporterConfigs) == 0 {
83+
return nil
84+
}
85+
86+
disableMetrics := map[string]bool{
87+
k8sNodeCPUUtilization: true,
88+
k8sPodCPUUtilization: true,
89+
containerCPUUtilization: true,
90+
}
91+
92+
// Check if the metrics are explicitly included in signalfx exporter.
93+
// If they are not included, we will disable them in kubeletstats receiver.
94+
for _, cm := range sfxExporterConfigs {
95+
cfg := signalfxExporterConfig{}
96+
err = confmap.NewFromStringMap(cm).Unmarshal(&cfg, confmap.WithIgnoreUnused())
97+
if err != nil {
98+
return nil // Ignore invalid config. Rely on the config validation to catch this.
99+
}
100+
if len(cfg.ExcludeMetrics) == 0 {
101+
// Apply default excluded metrics if not explicitly set.
102+
// https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/f2d8efe507083b0f38b6567f8dba3f37053bfa86/exporter/signalfxexporter/internal/translation/default_metrics.go#L133
103+
cfg.ExcludeMetrics = []dpfilters.MetricFilter{
104+
{MetricNames: []string{"/^(?i:(container)|(k8s\\.node)|(k8s\\.pod))\\.cpu\\.utilization$/"}},
105+
}
106+
}
107+
108+
var filter *dpfilters.FilterSet
109+
filter, err = dpfilters.NewFilterSet(cfg.ExcludeMetrics, cfg.IncludeMetrics)
110+
if err != nil {
111+
return nil // Ignore invalid config. Rely on the config validation to catch this.
112+
}
113+
for metricName := range disableMetrics {
114+
if !filter.Matches(&sfxpb.DataPoint{Metric: metricName}) {
115+
disableMetrics[metricName] = false
116+
}
117+
}
118+
}
119+
120+
// Disable the metrics in kubeletstats receiver.
121+
for receiverName, cfg := range kubeletReceiverConfigs {
122+
metricsCfg := map[string]any{}
123+
if cfg["metrics"] != nil {
124+
if v, ok := cfg["metrics"].(map[string]any); ok {
125+
metricsCfg = v
126+
}
127+
}
128+
if _, ok := metricsCfg[k8sNodeCPUUtilization]; !ok && disableMetrics[k8sNodeCPUUtilization] {
129+
metricsCfg[k8sNodeCPUUtilization] = map[string]any{"enabled": false}
130+
}
131+
if _, ok := metricsCfg[k8sPodCPUUtilization]; !ok && disableMetrics[k8sPodCPUUtilization] {
132+
metricsCfg[k8sPodCPUUtilization] = map[string]any{"enabled": false}
133+
}
134+
if _, ok := metricsCfg[containerCPUUtilization]; !ok && disableMetrics[containerCPUUtilization] {
135+
metricsCfg[containerCPUUtilization] = map[string]any{"enabled": false}
136+
}
137+
metricsCfgKey := fmt.Sprintf("receivers::%s::metrics", receiverName)
138+
if len(metricsCfg) > 0 {
139+
if err = cfgMap.Merge(confmap.NewFromStringMap(map[string]any{metricsCfgKey: metricsCfg})); err != nil {
140+
return err
141+
}
142+
}
143+
}
144+
145+
return nil
146+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// Copyright Splunk, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package configconverter
16+
17+
import (
18+
"context"
19+
"testing"
20+
21+
"github.com/stretchr/testify/assert"
22+
"github.com/stretchr/testify/require"
23+
"go.opentelemetry.io/collector/confmap/confmaptest"
24+
)
25+
26+
func TestDisableKubeletUtilizationMetrics(t *testing.T) {
27+
tests := []struct {
28+
name string
29+
input string
30+
wantOutput string
31+
}{
32+
{
33+
name: "no_kubeletstats_receiver",
34+
input: "testdata/disable_kubelet_utilization_metrics/no_kubeletstats_receiver.yaml",
35+
wantOutput: "testdata/disable_kubelet_utilization_metrics/no_kubeletstats_receiver.yaml",
36+
},
37+
{
38+
name: "no_signalfx_exporter",
39+
input: "testdata/disable_kubelet_utilization_metrics/no_signalfx_exporter.yaml",
40+
wantOutput: "testdata/disable_kubelet_utilization_metrics/no_signalfx_exporter.yaml",
41+
},
42+
{
43+
name: "disable_all_metrics",
44+
input: "testdata/disable_kubelet_utilization_metrics/disable_all_metrics_input.yaml",
45+
wantOutput: "testdata/disable_kubelet_utilization_metrics/disable_all_metrics_output.yaml",
46+
},
47+
{
48+
name: "do_not_change_enabled_metrics",
49+
input: "testdata/disable_kubelet_utilization_metrics/do_not_change_enabled_metrics_input.yaml",
50+
wantOutput: "testdata/disable_kubelet_utilization_metrics/do_not_change_enabled_metrics_output.yaml",
51+
},
52+
{
53+
name: "all_metrics_included_in_signalfx_exporter",
54+
input: "testdata/disable_kubelet_utilization_metrics/all_metrics_included_in_signalfx_exporter.yaml",
55+
wantOutput: "testdata/disable_kubelet_utilization_metrics/all_metrics_included_in_signalfx_exporter.yaml",
56+
},
57+
{
58+
name: "partially_excluded_in_signalfx_exporter",
59+
input: "testdata/disable_kubelet_utilization_metrics/partially_excluded_in_signalfx_exporter_input.yaml",
60+
wantOutput: "testdata/disable_kubelet_utilization_metrics/partially_excluded_in_signalfx_exporter_output.yaml",
61+
},
62+
{
63+
name: "partially_included_in_signalfx_exporter",
64+
input: "testdata/disable_kubelet_utilization_metrics/partially_included_in_signalfx_exporter_input.yaml",
65+
wantOutput: "testdata/disable_kubelet_utilization_metrics/partially_included_in_signalfx_exporter_output.yaml",
66+
},
67+
}
68+
for _, tt := range tests {
69+
t.Run(tt.name, func(t *testing.T) {
70+
expectedCfgMap, err := confmaptest.LoadConf(tt.wantOutput)
71+
require.NoError(t, err)
72+
require.NotNil(t, expectedCfgMap)
73+
74+
cfgMap, err := confmaptest.LoadConf(tt.input)
75+
require.NoError(t, err)
76+
require.NotNil(t, cfgMap)
77+
78+
err = DisableKubeletUtilizationMetrics{}.Convert(context.Background(), cfgMap)
79+
require.NoError(t, err)
80+
81+
assert.Equal(t, expectedCfgMap, cfgMap)
82+
})
83+
}
84+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
// Copyright The OpenTelemetry Authors
2+
// SPDX-License-Identifier: Apache-2.0
3+
// Temporary copied from the SignalFx exporter to be used in DisableKubeletUtilizationMetrics converter.
4+
5+
package dpfilters // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/translation/dpfilters"
6+
7+
import (
8+
"errors"
9+
10+
sfxpb "github.com/signalfx/com_signalfx_metrics_protobuf/model"
11+
)
12+
13+
type dataPointFilter struct {
14+
metricFilter *StringFilter
15+
dimensionsFilter *dimensionsFilter
16+
}
17+
18+
// newDataPointFilter returns a new dataPointFilter filter with the given configuration.
19+
func newDataPointFilter(metricNames []string, dimSet map[string][]string) (*dataPointFilter, error) {
20+
var metricFilter *StringFilter
21+
if len(metricNames) > 0 {
22+
var err error
23+
metricFilter, err = NewStringFilter(metricNames)
24+
if err != nil {
25+
return nil, err
26+
}
27+
}
28+
29+
var dimensionsFilter *dimensionsFilter
30+
if len(dimSet) > 0 {
31+
var err error
32+
dimensionsFilter, err = newDimensionsFilter(dimSet)
33+
if err != nil {
34+
return nil, err
35+
}
36+
}
37+
38+
if metricFilter == nil && dimensionsFilter == nil {
39+
return nil, errors.New("metric filter must have at least one metric or dimension defined on it")
40+
}
41+
42+
return &dataPointFilter{
43+
metricFilter: metricFilter,
44+
dimensionsFilter: dimensionsFilter,
45+
}, nil
46+
}
47+
48+
// Matches tests a datapoint to see whether it is excluded by this
49+
func (f *dataPointFilter) Matches(dp *sfxpb.DataPoint) bool {
50+
metricNameMatched := f.metricFilter == nil || f.metricFilter.Matches(dp.Metric)
51+
if metricNameMatched {
52+
return f.dimensionsFilter == nil || f.dimensionsFilter.Matches(dp.Dimensions)
53+
}
54+
return false
55+
56+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
// Copyright The OpenTelemetry Authors
2+
// SPDX-License-Identifier: Apache-2.0
3+
// Temporary copied from the SignalFx exporter to be used in DisableKubeletUtilizationMetrics converter.
4+
5+
package dpfilters // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/translation/dpfilters"
6+
7+
import (
8+
"errors"
9+
10+
sfxpb "github.com/signalfx/com_signalfx_metrics_protobuf/model"
11+
)
12+
13+
type dimensionsFilter struct {
14+
filterMap map[string]*StringFilter
15+
}
16+
17+
// newDimensionsFilter returns a filter that matches against a
18+
// sfxpb.Dimension slice. The filter will return false if there's
19+
// at least one dimension in the slice that fails to match. In case`
20+
// there are no filters for any of the dimension keys in the slice,
21+
// the filter will return false.
22+
func newDimensionsFilter(m map[string][]string) (*dimensionsFilter, error) {
23+
filterMap := map[string]*StringFilter{}
24+
for k := range m {
25+
if len(m[k]) == 0 {
26+
return nil, errors.New("string map value in filter cannot be empty")
27+
}
28+
29+
var err error
30+
filterMap[k], err = NewStringFilter(m[k])
31+
if err != nil {
32+
return nil, err
33+
}
34+
}
35+
36+
return &dimensionsFilter{
37+
filterMap: filterMap,
38+
}, nil
39+
}
40+
41+
func (f *dimensionsFilter) Matches(dimensions []*sfxpb.Dimension) bool {
42+
if len(dimensions) == 0 {
43+
return false
44+
}
45+
46+
var atLeastOneMatchedDimension bool
47+
for _, dim := range dimensions {
48+
dimF := f.filterMap[dim.Key]
49+
// Skip if there are no filters associated with current dimension key.
50+
if dimF == nil {
51+
continue
52+
}
53+
54+
if !dimF.Matches(dim.Value) {
55+
return false
56+
}
57+
58+
if !atLeastOneMatchedDimension {
59+
atLeastOneMatchedDimension = true
60+
}
61+
}
62+
63+
return atLeastOneMatchedDimension
64+
}

0 commit comments

Comments
 (0)