|
| 1 | +--- |
| 2 | +title: "Native Histograms" |
| 3 | +linkTitle: "Native Histograms" |
| 4 | +weight: 10 |
| 5 | +slug: native-histograms |
| 6 | +--- |
| 7 | + |
| 8 | +## Context |
| 9 | +The Prometheus introduces a native histogram, a new sample type, to address several problems that the classic histograms (originally implemented histograms) have. |
| 10 | +Please read to [Prometheus Native Histograms Document](https://prometheus.io/docs/specs/native_histograms/) if more detailed information is necessary. |
| 11 | + |
| 12 | +This guide explains how to configure the native histograms on the Cortex. |
| 13 | + |
| 14 | +## How to configure native histograms |
| 15 | +This section explains how to configure native histograms on the Cortex. |
| 16 | +### Enable Ingestion |
| 17 | +To ingest native histogram ingestion, set the flag `-blocks-storage.tsdb.enable-native-histograms`. |
| 18 | + |
| 19 | +And via yaml: |
| 20 | +```yaml |
| 21 | +blocks_storage: |
| 22 | + tsdb: |
| 23 | + enable_native_histograms: <bool> |
| 24 | +``` |
| 25 | +By default, it is disabled, so make sure to enable it if you need native histograms for your system monitoring. |
| 26 | +
|
| 27 | +When a `write request` contains both `samples` and `native histogram`, the ingestion behavior is as follows: |
| 28 | +- When it is enabled, the Cortex ingests samples and native histograms. If ingestion is successful, the Cortex returns a 200 status code. |
| 29 | +- When it is disabled, the Cortex ingests samples but discards the native histograms. If sample ingestion is successful, the Cortex returns a 200 status code. |
| 30 | + |
| 31 | +You can track the discarded number of native histograms via the `cortex_discarded_samples_total{reason="native-histogram-sample"}` promQL. |
| 32 | + |
| 33 | +### Set Bucket limit |
| 34 | +To limit the number of buckets(= sum of the number of positive and negative buckets) native histograms ingested, set the flag `-validation.max-native-histogram-buckets`. |
| 35 | + |
| 36 | +And via yaml: |
| 37 | +```yaml |
| 38 | +limits: |
| 39 | + max_native_histogram_buckets: <int> |
| 40 | +``` |
| 41 | + |
| 42 | +The default value is 0, which means no limit. If the total number of positive and negative buckets exceeds the limit, the Cortex emits a validation error with status code 400 (Bad Request). |
| 43 | + |
| 44 | +To limit the number of buckets per tenant, you can utilize a [runtime config](../configuration/arguments.md#runtime-configuration-file). |
| 45 | + |
| 46 | +For example, the following yaml file specifies the number of bucket limit 160 for `user-1` and no limit for `user-2`. |
| 47 | + |
| 48 | +``` |
| 49 | +overrides: |
| 50 | + user-1: |
| 51 | + max_native_histogram_buckets: 160 |
| 52 | + user-2: |
| 53 | + max_native_histogram_buckets: 0 |
| 54 | +``` |
| 55 | +
|
| 56 | +## How to enable out-of-order native histograms ingestion |
| 57 | +Like samples out-of-order ingestion, the Cortex allows out-of-order ingestion for the native histogram. |
| 58 | +To enable it, set the flag `-ingester.enable-ooo-native-histograms`. |
| 59 | +
|
| 60 | +And via yaml: |
| 61 | +```yaml |
| 62 | +limits: |
| 63 | + enable_ooo_native_histograms: <bool> |
| 64 | +``` |
| 65 | + |
| 66 | +Is it only works if when `-blocks-storage.tsdb.enable-native-histograms=true` and `-ingester.out-of-order-time-window > 0`. |
| 67 | + |
| 68 | +To enable it per tenant, you can utilize a [runtime config](../configuration/arguments.md#runtime-configuration-file). |
| 69 | + |
| 70 | +For example, the following yaml file specifies enabling out-of-order native histogram ingestion for `user-1`, but not for `user-2`. |
| 71 | + |
| 72 | +``` |
| 73 | +overrides: |
| 74 | + user-1: |
| 75 | + enable_ooo_native_histograms: true |
| 76 | + user-2: |
| 77 | + enable_ooo_native_histograms: false |
| 78 | +``` |
| 79 | + |
| 80 | +**Caution**: It is applied after the Ingester restart if it is changed at runtime through the runtime config. |
| 81 | +For example, if you have changed the `enable_ooo_native_histograms` value to `false` of the `user-1` via the below yaml file, then the Ingester stops the out-of-order ingestion not until the Ingester restarts. |
| 82 | + |
| 83 | +``` |
| 84 | +overrides: |
| 85 | + user-1: |
| 86 | + enable_ooo_native_histograms: false |
| 87 | + user-2: |
| 88 | + enable_ooo_native_histograms: false |
| 89 | +``` |
0 commit comments