Skip to content

Commit c6dc6b4

Browse files
authored
Add docs for native histogram (#6635)
1 parent 0d78ead commit c6dc6b4

File tree

1 file changed

+89
-0
lines changed

1 file changed

+89
-0
lines changed

docs/guides/native-histograms.md

+89
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: "Native Histograms"
3+
linkTitle: "Native Histograms"
4+
weight: 10
5+
slug: native-histograms
6+
---
7+
8+
## Context
9+
The Prometheus introduces a native histogram, a new sample type, to address several problems that the classic histograms (originally implemented histograms) have.
10+
Please read to [Prometheus Native Histograms Document](https://prometheus.io/docs/specs/native_histograms/) if more detailed information is necessary.
11+
12+
This guide explains how to configure the native histograms on the Cortex.
13+
14+
## How to configure native histograms
15+
This section explains how to configure native histograms on the Cortex.
16+
### Enable Ingestion
17+
To ingest native histogram ingestion, set the flag `-blocks-storage.tsdb.enable-native-histograms`.
18+
19+
And via yaml:
20+
```yaml
21+
blocks_storage:
22+
tsdb:
23+
enable_native_histograms: <bool>
24+
```
25+
By default, it is disabled, so make sure to enable it if you need native histograms for your system monitoring.
26+
27+
When a `write request` contains both `samples` and `native histogram`, the ingestion behavior is as follows:
28+
- When it is enabled, the Cortex ingests samples and native histograms. If ingestion is successful, the Cortex returns a 200 status code.
29+
- When it is disabled, the Cortex ingests samples but discards the native histograms. If sample ingestion is successful, the Cortex returns a 200 status code.
30+
31+
You can track the discarded number of native histograms via the `cortex_discarded_samples_total{reason="native-histogram-sample"}` promQL.
32+
33+
### Set Bucket limit
34+
To limit the number of buckets(= sum of the number of positive and negative buckets) native histograms ingested, set the flag `-validation.max-native-histogram-buckets`.
35+
36+
And via yaml:
37+
```yaml
38+
limits:
39+
max_native_histogram_buckets: <int>
40+
```
41+
42+
The default value is 0, which means no limit. If the total number of positive and negative buckets exceeds the limit, the Cortex emits a validation error with status code 400 (Bad Request).
43+
44+
To limit the number of buckets per tenant, you can utilize a [runtime config](../configuration/arguments.md#runtime-configuration-file).
45+
46+
For example, the following yaml file specifies the number of bucket limit 160 for `user-1` and no limit for `user-2`.
47+
48+
```
49+
overrides:
50+
user-1:
51+
max_native_histogram_buckets: 160
52+
user-2:
53+
max_native_histogram_buckets: 0
54+
```
55+
56+
## How to enable out-of-order native histograms ingestion
57+
Like samples out-of-order ingestion, the Cortex allows out-of-order ingestion for the native histogram.
58+
To enable it, set the flag `-ingester.enable-ooo-native-histograms`.
59+
60+
And via yaml:
61+
```yaml
62+
limits:
63+
enable_ooo_native_histograms: <bool>
64+
```
65+
66+
Is it only works if when `-blocks-storage.tsdb.enable-native-histograms=true` and `-ingester.out-of-order-time-window > 0`.
67+
68+
To enable it per tenant, you can utilize a [runtime config](../configuration/arguments.md#runtime-configuration-file).
69+
70+
For example, the following yaml file specifies enabling out-of-order native histogram ingestion for `user-1`, but not for `user-2`.
71+
72+
```
73+
overrides:
74+
user-1:
75+
enable_ooo_native_histograms: true
76+
user-2:
77+
enable_ooo_native_histograms: false
78+
```
79+
80+
**Caution**: It is applied after the Ingester restart if it is changed at runtime through the runtime config.
81+
For example, if you have changed the `enable_ooo_native_histograms` value to `false` of the `user-1` via the below yaml file, then the Ingester stops the out-of-order ingestion not until the Ingester restarts.
82+
83+
```
84+
overrides:
85+
user-1:
86+
enable_ooo_native_histograms: false
87+
user-2:
88+
enable_ooo_native_histograms: false
89+
```

0 commit comments

Comments
 (0)