[k8s] Define semantic conventions for k8s resource quota metrics

[ChrsMark]

Propose new conventions

Area(s)

area:k8s

Is your change request related to a problem? Please describe.

As part of the K8s SemConv stability work we need to define k8s resource quota related metrics that are already in use by the Opentelemetry Collector.

Related to #1032

Describe the solution you'd like

The existing metrics in use that we need to define as semantic conventions are the following:

k8s.resource_quota.hard_limit
k8s.resource_quota.used

[ChrsMark]

A ResourceQuota object will come with a status like the following:

status:
  hard:
    limits.cpu: "2"
    limits.memory: 2Gi
    requests.cpu: "1"
    requests.memory: 1Gi
  used:
    limits.cpu: 800m
    limits.memory: 800Mi
    requests.cpu: 400m
    requests.memory: 600Mi

(ref)

At https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.123.0/receiver/k8sclusterreceiver/internal/resourcequota/resourcequotas.go#L15-L36 we retrieve all those but instead of emitting 8 different metrics we only emit 2 with 4 attributes each. Example:

Resource SchemaURL: https://opentelemetry.io/schemas/1.18.0
Resource attributes:
     -> k8s.resourcequota.uid: Str(31c6317a-d8d4-48ea-adb0-e35074818623)
     -> k8s.resourcequota.name: Str(mem-cpu-demo)
     -> k8s.namespace.name: Str(default)
ScopeMetrics #0
ScopeMetrics SchemaURL:
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver 0.123.0-dev
Metric #0
Descriptor:
     -> Name: k8s.resource_quota.hard_limit
     -> Description: The upper limit for a particular resource in a specific namespace. Will only be sent if a quota is specified. CPU requests/limits will be sent as millicores
     -> Unit: {resource}
     -> DataType: Gauge
NumberDataPoints #0
Data point attributes:
     -> resource: Str(limits.cpu)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 2000
NumberDataPoints #1
Data point attributes:
     -> resource: Str(limits.memory)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 2147483648
NumberDataPoints #2
Data point attributes:
     -> resource: Str(requests.cpu)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1000
NumberDataPoints #3
Data point attributes:
     -> resource: Str(requests.memory)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1073741824
Metric #1
Descriptor:
     -> Name: k8s.resource_quota.used
     -> Description: The usage for a particular resource in a specific namespace. Will only be sent if a quota is specified. CPU requests/limits will be sent as millicores
     -> Unit: {resource}
     -> DataType: Gauge
NumberDataPoints #0
Data point attributes:
     -> resource: Str(limits.cpu)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1000
NumberDataPoints #1
Data point attributes:
     -> resource: Str(limits.memory)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1073741824
NumberDataPoints #2
Data point attributes:
     -> resource: Str(requests.cpu)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1000
NumberDataPoints #3
Data point attributes:
     -> resource: Str(requests.memory)
StartTimestamp: 2025-04-14 09:34:35.770466928 +0000 UTC
Timestamp: 2025-04-14 09:34:45.968422544 +0000 UTC
Value: 1073741824

This is quite weird and does not seem to follow the SemConv guidelines. Shouldn't we have 8 different metrics instead?

i.e.:

k8s.resource_quota.hard.cpu_limit
k8s.resource_quota.hard.cpu_request
k8s.resource_quota.hard.memory_limit
k8s.resource_quota.hard.memory_request

k8s.resource_quota.used.cpu_limit
k8s.resource_quota.used.cpu_request
k8s.resource_quota.used.memory_limit
k8s.resource_quota.used.memory_request

I'm not 100% sure if hard and used should be namespaces or not.
@open-telemetry/semconv-k8s-approvers I'd love your feedback here.

[jinja2]

Resource quotas aren't limited to cpu and memory. It can be set for storage, gpu, object counts (k8s native and custom resources), etc. Here's an example -

apiVersion: v1
kind: ResourceQuota
metadata:
  creationTimestamp: "2024-04-03T19:16:22Z"
  name: default-resource-quotas
  namespace: istio-ingress
  resourceVersion: "242026"
  uid: <uid>
spec:
  hard:
    count/gateways.networking.istio.io: "20"
    count/targetgroupbindings.elbv2.k8s.aws: "50"
    services.loadbalancers: "10"
    services.nodeports: "30"
status:
  hard:
    count/gateways.networking.istio.io: "20"
    count/targetgroupbindings.elbv2.k8s.aws: "50"
    services.loadbalancers: "10"
    services.nodeports: "30"
  used:
    count/gateways.networking.istio.io: "10"
    count/targetgroupbindings.elbv2.k8s.aws: "17"
    services.loadbalancers: "6"
    services.nodeports: "15"

Given the wide range of possible resource types, the existing pattern of using an attribute to indicate the resource type seems more scalable, and flexible to me.

[ChrsMark]

Thank's @jinja2 for the note, I guess that might was the idea for it being implemented like this in the Collector.

However, I'm not sure this design can be covered in SemConv. Can the same metric be of different type/unit based on its attributes? For instance, memory quotas will be measured in Bytes while cpu ones in cores, right?

I wonder if scoping each resource under its namespace makes more sense here, potentially starting from the most common ones?