Skip to content

Commit 68252cf

Browse files
authored
Merge pull request #6200 from SungJin1212/Add-support-rule-evalution-via-query-frontend-docs
docs:Add support rule evaluation via query frontend docs
2 parents c685cf4 + f4e68de commit 68252cf

File tree

1 file changed

+47
-0
lines changed

1 file changed

+47
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: "Rule evaluations via query frontend"
3+
linkTitle: "Rule evaluations via query frontend"
4+
weight: 10
5+
slug: rule-evalutions-via-query-frontend
6+
---
7+
8+
This guide explains how to configure the Ruler to evaluate rules via Query Frontends instead of the Ingester/Store Gateway, and the pros and cons of rule evaluation via Query Frontend.
9+
10+
## How to enable
11+
12+
By default, the Ruler queries both Ingesters and Store Gateway depending on the Rule time range for evaluating rules (alerting rules or recording rules). If you have set `-ruler.frontend-address`, then the Ruler queries the Query Frontend for evaluation rules.
13+
The address should be the gRPC listen address in host:port format.
14+
15+
16+
You can configure via args:
17+
```
18+
-ruler.frontend-address=query-frontend.svc.cluster.local:9095
19+
```
20+
And via yaml:
21+
```yaml
22+
ruler:
23+
frontend_address: query-frontend.svc.cluster.local:9095
24+
```
25+
26+
In addition, you can configure gRPC client (Ruler -> Query Frontend) config, please refer to frontend_client section in [ruler config](../configuration/config-file-reference.md#ruler_config).
27+
28+
## Configure query response format
29+
You can configure the query response format via `-ruler.query-response-format`. It is used to retrieve query results from the Query Frontend.
30+
The supported values are `protobuf` and `json`. We recommend using `protobuf`(default) because the retrieved query results containing native histograms are only supported on `protobuf`.
31+
32+
33+
## Pros and Cons
34+
If this feature is enabled, the query execute path is as follows:
35+
36+
Ruler -> Query Frontend -> Query Scheduler -> Querier -> Ingester/Store Gateway
37+
38+
There are pros and cons regarding query performance as more hops than before (Ruler -> Ingester/Store Gateway).
39+
### Pros
40+
- The rule evaluation performance could be improved in such a situation where the number of Queriers pulling queries from the Query Scheduler is good enough.
41+
If then, the queries in Query Scheduler are fetched in a reasonable time (i.e. a lot of hops are not a defect for query performance). In this environment, query performance could be improved as we can use Query Frontend features like the vertical query sharding.
42+
- The Ruler can use fewer resources as it doesn't need to run a query engine to evaluate rules.
43+
44+
### Cons
45+
- If there are not enough Queriers, adding rule queries to Query Scheduler could cause query starvation problem (queries in Query Scheduler could not be fetched in a reasonable time), so rules cannot be evaluated in time.
46+
47+
You can utilize the `cortex_prometheus_rule_evaluation_duration_seconds` metric whether to use `-ruler.frontend-address`.

0 commit comments

Comments
 (0)