Skip to content

Commit 98c54ad

Browse files
authored
chore(observability): remove deprecated internal metrics + massive cleanup to vector top and graphql API (vectordotdev#17516)
## Context This PR is primarily, and simply, focused on one thing: removing now deprecated internal metrics. Specifically: - `events_in_total` - `events_out_total` - `processed_bytes_total` - `processed_events_total` These metrics have been deprecated for a while, replaced by the metrics defined in the [Component Specification](). For example, `component_sent_events_total` replaced `events_out_total`, and `processed_bytes_total` is replaced by `component_sent_bytes_total`/`component_received_bytes_total`: no more having to remember what direction the metric refers to based on what type of component is emitting it. ## Solution The change to simply stop emitting these deprecated metrics is straightforward, but most of the work in this PR centered around updating the GraphQL API, and subsequently `vector top`, to switch from the `processed_*` metrics to their `component_*` counterparts. While in the area, I adjusted some of the `vector top` output to display separated bytes in/bytes out, similar to the existing events in/events out split. This also involved a small amount of work to adjust the layout constraints, and so on, to accommodate for that. Additionally, many updates were made to the component Cue files to remove references to these now-removed metrics. ## Reviewer Notes ### Dichotomy between `processed_bytes_total` and the `component`-prefixed replacements Prior to this PR, we were backfilling `processed_bytes_total` in the following way: - for sources, alias `component_received_bytes_total` to `processed_bytes_total` - for sinks, alias `component_sent_bytes_total` to `processed_bytes_total` As such, we never emitted this metric for transforms, and `processed_bytes_total` was an analogue for network bytes. A lot of the work has tried to reflect that, such as in the GraphQL API, where spots that had `processed_bytes` are now replaced with the component type-relevant value, being either `received_bytes` or `sent_bytes`. Closes vectordotdev#9314. Closes vectordotdev#7346.
1 parent a551f33 commit 98c54ad

File tree

121 files changed

+1134
-2147
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

121 files changed

+1134
-2147
lines changed

lib/vector-api-client/graphql/queries/components.graphql

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,8 @@ query ComponentsQuery($first: Int!) {
1414
}
1515
metrics {
1616
__typename
17-
processedEventsTotal {
18-
processedEventsTotal
19-
}
20-
processedBytesTotal {
21-
processedBytesTotal
17+
receivedBytesTotal {
18+
receivedBytesTotal
2219
}
2320
receivedEventsTotal {
2421
receivedEventsTotal
@@ -37,12 +34,6 @@ query ComponentsQuery($first: Int!) {
3734
}
3835
metrics {
3936
__typename
40-
processedEventsTotal {
41-
processedEventsTotal
42-
}
43-
processedBytesTotal {
44-
processedBytesTotal
45-
}
4637
receivedEventsTotal {
4738
receivedEventsTotal
4839
}
@@ -54,15 +45,12 @@ query ComponentsQuery($first: Int!) {
5445
... on Sink {
5546
metrics {
5647
__typename
57-
processedEventsTotal {
58-
processedEventsTotal
59-
}
60-
processedBytesTotal {
61-
processedBytesTotal
62-
}
6348
receivedEventsTotal {
6449
receivedEventsTotal
6550
}
51+
sentBytesTotal {
52+
sentBytesTotal
53+
}
6654
sentEventsTotal {
6755
sentEventsTotal
6856
}

0 commit comments

Comments
 (0)