Skip to content

Commit ba54132

Browse files
alladazbirenbaum
authored andcommitted
[Refactor] Complete metrics overhaul (TraceMachina#1192)
Metrics got an entire overhaul. Instead of relying on a broken prometheus library to publish our metrics, we now use the `tracing` library and with OpenTelemetry that we bind together then publish into a prometheus library. Metrics are now mostly derive-macros. This means that the struct can express what it wants to export and a help text. The library will choose if it is able to export it. Tracing now works by calling `.publish()` on the parent structs, those structs need to call `.publish()` on all the child members it wishes to publish data about. If a "group" is requested, use the `group!()` macro, which under-the-hood calls `tracing::span` with some special labels. At primitive layers, it will call the `publish!()` macro, which will call `tracing::event!()` macro under-the-hood with some special fields set. A custom `tracing::Subscriber` will intercept all the events and spans and convert them into a json-like object. This object can then be exported as real json or encoded into other formats like otel/prometheus. closes: TraceMachina#1164, TraceMachina#650, TraceMachina#384, TraceMachina#209 towards: TraceMachina#206
1 parent 50b51ac commit ba54132

File tree

80 files changed

+2722
-1522
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+2722
-1522
lines changed

BUILD.bazel

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ rust_binary(
1616
deps = [
1717
"//nativelink-config",
1818
"//nativelink-error",
19+
"//nativelink-metric",
20+
"//nativelink-metric-collector",
1921
"//nativelink-proto",
2022
"//nativelink-scheduler",
2123
"//nativelink-service",
@@ -28,16 +30,21 @@ rust_binary(
2830
"@crates//:futures",
2931
"@crates//:hyper",
3032
"@crates//:mimalloc",
33+
"@crates//:opentelemetry",
34+
"@crates//:opentelemetry-prometheus",
35+
"@crates//:opentelemetry_sdk",
3136
"@crates//:parking_lot",
32-
"@crates//:prometheus-client",
37+
"@crates//:prometheus",
3338
"@crates//:rustls-pemfile",
3439
"@crates//:scopeguard",
40+
"@crates//:serde_json",
3541
"@crates//:serde_json5",
3642
"@crates//:tokio",
3743
"@crates//:tokio-rustls",
3844
"@crates//:tonic",
3945
"@crates//:tower",
4046
"@crates//:tracing",
47+
"@crates//:tracing-subscriber",
4148
],
4249
)
4350

0 commit comments

Comments
 (0)