Skip to content

Commit d6e9820

Browse files
zyy17zyy17
authored
chore: add logs dashboard (#6028)
1 parent 7b4df63 commit d6e9820

File tree

11 files changed

+347
-21
lines changed

11 files changed

+347
-21
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ debug/
2828
# Logs
2929
**/__unittest_logs
3030
logs/
31+
!grafana/dashboards/logs/
3132

3233
# cpython's generated python byte code
3334
**/__pycache__/

‎grafana/README.md

Lines changed: 50 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,30 +2,63 @@
22

33
## Overview
44

5-
This repository maintains the Grafana dashboards for GreptimeDB. It has two types of dashboards:
5+
This repository contains Grafana dashboards for visualizing metrics and logs of GreptimeDB instances running in either cluster or standalone mode. **The Grafana version should be greater than 9.0**.
66

7-
- `cluster/dashboard.json`: The Grafana dashboard for the GreptimeDB cluster. Read the [dashboard.md](./dashboards/cluster/dashboard.md) for more details.
8-
- `standalone/dashboard.json`: The Grafana dashboard for the standalone GreptimeDB instance. **It's generated from the `cluster/dashboard.json` by removing the instance filter through the `make dashboards` command**. Read the [dashboard.md](./dashboards/standalone/dashboard.md) for more details.
7+
We highly recommend using the self-monitoring feature provided by [GreptimeDB Operator](https://github.com/GrepTimeTeam/greptimedb-operator) to automatically collect metrics and logs from your GreptimeDB instances and store them in a dedicated GreptimeDB instance.
98

10-
As the rapid development of GreptimeDB, the metrics may be changed, and please feel free to submit your feedback and/or contribution to this dashboard 🤗
9+
- **Metrics Dashboards**
1110

12-
**NOTE**:
11+
- `dashboards/metrics/cluster/dashboard.json`: The Grafana dashboard for the GreptimeDB cluster. Read the [dashboard.md](./dashboards/metrics/cluster/dashboard.md) for more details.
12+
13+
- `dashboards/metrics/standalone/dashboard.json`: The Grafana dashboard for the standalone GreptimeDB instance. **It's generated from the `cluster/dashboard.json` by removing the instance filter through the `make dashboards` command**. Read the [dashboard.md](./dashboards/metrics/standalone/dashboard.md) for more details.
1314

14-
- The Grafana version should be greater than 9.0.
15+
- **Logs Dashboard**
1516

16-
- If you want to modify the dashboards, you only need to modify the `cluster/dashboard.json` and run the `make dashboards` command to generate the `standalone/dashboard.json` and other related files.
17+
The `dashboards/logs/dashboard.json` provides a comprehensive Grafana dashboard for visualizing GreptimeDB logs. To utilize this dashboard effectively, you need to collect logs in JSON format from your GreptimeDB instances and store them in a dedicated GreptimeDB instance.
1718

18-
To maintain the dashboards easily, we use the [`dac`](https://github.com/zyy17/dac) tool to generate the intermediate dashboards and markdown documents:
19+
For proper integration, the logs table must adhere to the following schema design with the table name `_gt_logs`:
1920

20-
- `cluster/dashboard.yaml`: The intermediate dashboard for the GreptimeDB cluster.
21-
- `standalone/dashboard.yaml`: The intermediate dashboard for the standalone GreptimeDB instance.
21+
```sql
22+
CREATE TABLE IF NOT EXISTS `_gt_logs` (
23+
`pod_ip` STRING NULL,
24+
`namespace` STRING NULL,
25+
`cluster` STRING NULL,
26+
`file` STRING NULL,
27+
`module_path` STRING NULL,
28+
`level` STRING NULL,
29+
`target` STRING NULL,
30+
`role` STRING NULL,
31+
`pod` STRING NULL SKIPPING INDEX WITH(granularity = '10240', type = 'BLOOM'),
32+
`message` STRING NULL FULLTEXT INDEX WITH(analyzer = 'English', backend = 'bloom', case_sensitive = 'false'),
33+
`err` STRING NULL FULLTEXT INDEX WITH(analyzer = 'English', backend = 'bloom', case_sensitive = 'false'),
34+
`timestamp` TIMESTAMP(9) NOT NULL,
35+
TIME INDEX (`timestamp`),
36+
PRIMARY KEY (`level`, `target`, `role`)
37+
)
38+
ENGINE=mito
39+
WITH (
40+
append_mode = 'true'
41+
)
42+
```
43+
44+
## Development
45+
46+
As GreptimeDB evolves rapidly, metrics may change over time. We welcome your feedback and contributions to improve these dashboards 🤗
47+
48+
To modify the metrics dashboards, simply edit the `dashboards/metrics/cluster/dashboard.json` file and run the `make dashboards` command. This will automatically generate the updated `dashboards/metrics/standalone/dashboard.json` and other related files.
49+
50+
For easier dashboard maintenance, we utilize the [`dac`](https://github.com/zyy17/dac) tool to generate human-readable intermediate dashboards and documentation:
51+
52+
- `dashboards/metrics/cluster/dashboard.yaml`: The intermediate dashboard file for the GreptimeDB cluster.
53+
- `dashboards/metrics/standalone/dashboard.yaml`: The intermediate dashboard file for standalone GreptimeDB instances.
2254

2355
## Data Sources
2456

25-
There are two data sources for the dashboards to fetch the metrics:
57+
The following data sources are used to fetch metrics and logs:
2658

27-
- **Prometheus**: Expose the metrics of GreptimeDB.
28-
- **Information Schema**: It is the MySQL port of the current monitored instance. The `overview` dashboard will use this datasource to show the information schema of the current instance.
59+
- **`${metrics}`**: Prometheus data source for providing the GreptimeDB metrics.
60+
- **`${logs}`**: MySQL data source for providing the GreptimeDB logs.
61+
- **`${information_schema}`**: MySQL data source for providing the information schema of the current instance and used for the `overview` panel. It is the MySQL port of the current monitored instance.
2962

3063
## Instance Filters
3164

@@ -43,9 +76,9 @@ And the legend will be like: `[{{instance}}]-[{{ pod }}]`.
4376

4477
## Deployment
4578

46-
### Helm
79+
### (Recommended) Helm Chart
4780

48-
If you use the Helm [chart](https://github.com/GreptimeTeam/helm-charts) to deploy a GreptimeDB cluster, you can enable self-monitoring by setting the following values in your Helm chart:
81+
If you use the [Helm Chart](https://github.com/GreptimeTeam/helm-charts) to deploy a GreptimeDB cluster, you can enable self-monitoring by setting the following values in your Helm chart:
4982

5083
- `monitoring.enabled=true`: Deploys a standalone GreptimeDB instance dedicated to monitoring the cluster;
5184
- `grafana.enabled=true`: Deploys Grafana and automatically imports the monitoring dashboard;
@@ -85,5 +118,5 @@ The standalone GreptimeDB instance will collect metrics from your cluster, and t
85118
86119
3. **Import the dashboards based on your deployment scenario**
87120
88-
- **Cluster**: Import the `cluster/dashboard.json` dashboard.
89-
- **Standalone**: Import the `standalone/dashboard.json` dashboard.
121+
- **Cluster**: Import the `dashboards/metrics/cluster/dashboard.json` dashboard.
122+
- **Standalone**: Import the `dashboards/metrics/standalone/dashboard.json` dashboard.

‎grafana/dashboards/logs/dashboard.json

Lines changed: 292 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,292 @@
1+
{
2+
"annotations": {
3+
"list": [
4+
{
5+
"builtIn": 1,
6+
"datasource": {
7+
"type": "grafana",
8+
"uid": "-- Grafana --"
9+
},
10+
"enable": true,
11+
"hide": true,
12+
"iconColor": "rgba(0, 211, 255, 1)",
13+
"name": "Annotations & Alerts",
14+
"type": "dashboard"
15+
}
16+
]
17+
},
18+
"editable": true,
19+
"fiscalYearStartMonth": 0,
20+
"graphTooltip": 0,
21+
"id": 12,
22+
"links": [],
23+
"panels": [
24+
{
25+
"datasource": {
26+
"default": false,
27+
"type": "mysql",
28+
"uid": "${datasource}"
29+
},
30+
"fieldConfig": {
31+
"defaults": {},
32+
"overrides": []
33+
},
34+
"gridPos": {
35+
"h": 20,
36+
"w": 24,
37+
"x": 0,
38+
"y": 0
39+
},
40+
"id": 1,
41+
"options": {
42+
"dedupStrategy": "none",
43+
"enableInfiniteScrolling": true,
44+
"enableLogDetails": true,
45+
"prettifyLogMessage": false,
46+
"showCommonLabels": false,
47+
"showLabels": false,
48+
"showTime": true,
49+
"sortOrder": "Descending",
50+
"wrapLogMessage": false
51+
},
52+
"pluginVersion": "11.6.0",
53+
"targets": [
54+
{
55+
"dataset": "greptime_private",
56+
"datasource": {
57+
"type": "mysql",
58+
"uid": "${datasource}"
59+
},
60+
"editorMode": "code",
61+
"format": "table",
62+
"rawQuery": true,
63+
"rawSql": "SELECT `timestamp`, CONCAT('[', `level`, ']', ' ', '<', `target`, '>', ' ', `message`),\n `role`,\n `pod`,\n `pod_ip`,\n `namespace`,\n `cluster`,\n `err`,\n `file`,\n `module_path`\nFROM\n `_gt_logs`\nWHERE\n (\n \"$level\" = \"'all'\"\n OR `level` IN ($level)\n ) \n AND (\n \"$role\" = \"'all'\"\n OR `role` IN ($role)\n )\n AND (\n \"$pod\" = \"\"\n OR `pod` = '$pod'\n )\n AND (\n \"$target\" = \"\"\n OR `target` = '$target'\n )\n AND (\n \"$search\" = \"\"\n OR matches_term(`message`, '$search')\n )\n AND (\n \"$exclude\" = \"\"\n OR NOT matches_term(`message`, '$exclude')\n )\n AND $__timeFilter(`timestamp`)\nORDER BY `timestamp` DESC\nLIMIT $limit;\n",
64+
"refId": "A",
65+
"sql": {
66+
"columns": [
67+
{
68+
"parameters": [],
69+
"type": "function"
70+
}
71+
],
72+
"groupBy": [
73+
{
74+
"property": {
75+
"type": "string"
76+
},
77+
"type": "groupBy"
78+
}
79+
],
80+
"limit": 50
81+
}
82+
}
83+
],
84+
"title": "Logs",
85+
"type": "logs"
86+
}
87+
],
88+
"preload": false,
89+
"refresh": "",
90+
"schemaVersion": 41,
91+
"tags": [],
92+
"templating": {
93+
"list": [
94+
{
95+
"current": {
96+
"text": "logs",
97+
"value": "P98F38F12DB221A8C"
98+
},
99+
"includeAll": false,
100+
"name": "datasource",
101+
"options": [],
102+
"query": "mysql",
103+
"refresh": 1,
104+
"regex": "",
105+
"type": "datasource"
106+
},
107+
{
108+
"allValue": "'all'",
109+
"current": {
110+
"text": [
111+
"$__all"
112+
],
113+
"value": [
114+
"$__all"
115+
]
116+
},
117+
"includeAll": true,
118+
"label": "level",
119+
"multi": true,
120+
"name": "level",
121+
"options": [
122+
{
123+
"selected": false,
124+
"text": "INFO",
125+
"value": "INFO"
126+
},
127+
{
128+
"selected": false,
129+
"text": "ERROR",
130+
"value": "ERROR"
131+
},
132+
{
133+
"selected": false,
134+
"text": "WARN",
135+
"value": "WARN"
136+
},
137+
{
138+
"selected": false,
139+
"text": "DEBUG",
140+
"value": "DEBUG"
141+
},
142+
{
143+
"selected": false,
144+
"text": "TRACE",
145+
"value": "TRACE"
146+
}
147+
],
148+
"query": "INFO,ERROR,WARN,DEBUG,TRACE",
149+
"type": "custom"
150+
},
151+
{
152+
"allValue": "'all'",
153+
"current": {
154+
"text": [
155+
"$__all"
156+
],
157+
"value": [
158+
"$__all"
159+
]
160+
},
161+
"includeAll": true,
162+
"label": "role",
163+
"multi": true,
164+
"name": "role",
165+
"options": [
166+
{
167+
"selected": false,
168+
"text": "datanode",
169+
"value": "datanode"
170+
},
171+
{
172+
"selected": false,
173+
"text": "frontend",
174+
"value": "frontend"
175+
},
176+
{
177+
"selected": false,
178+
"text": "meta",
179+
"value": "meta"
180+
}
181+
],
182+
"query": "datanode,frontend,meta",
183+
"type": "custom"
184+
},
185+
{
186+
"current": {
187+
"text": "",
188+
"value": ""
189+
},
190+
"label": "pod",
191+
"name": "pod",
192+
"options": [
193+
{
194+
"selected": true,
195+
"text": "",
196+
"value": ""
197+
}
198+
],
199+
"query": "",
200+
"type": "textbox"
201+
},
202+
{
203+
"current": {
204+
"text": "",
205+
"value": ""
206+
},
207+
"label": "target",
208+
"name": "target",
209+
"options": [
210+
{
211+
"selected": true,
212+
"text": "",
213+
"value": ""
214+
}
215+
],
216+
"query": "",
217+
"type": "textbox"
218+
},
219+
{
220+
"current": {
221+
"text": "",
222+
"value": ""
223+
},
224+
"label": "search",
225+
"name": "search",
226+
"options": [
227+
{
228+
"selected": true,
229+
"text": "",
230+
"value": ""
231+
}
232+
],
233+
"query": "",
234+
"type": "textbox"
235+
},
236+
{
237+
"current": {
238+
"text": "",
239+
"value": ""
240+
},
241+
"label": "exclude",
242+
"name": "exclude",
243+
"options": [
244+
{
245+
"selected": true,
246+
"text": "",
247+
"value": ""
248+
}
249+
],
250+
"query": "",
251+
"type": "textbox"
252+
},
253+
{
254+
"current": {
255+
"text": "2000",
256+
"value": "2000"
257+
},
258+
"includeAll": false,
259+
"label": "limit",
260+
"name": "limit",
261+
"options": [
262+
{
263+
"selected": true,
264+
"text": "2000",
265+
"value": "2000"
266+
},
267+
{
268+
"selected": false,
269+
"text": "5000",
270+
"value": "5000"
271+
},
272+
{
273+
"selected": false,
274+
"text": "8000",
275+
"value": "8000"
276+
}
277+
],
278+
"query": "2000,5000,8000",
279+
"type": "custom"
280+
}
281+
]
282+
},
283+
"time": {
284+
"from": "now-6h",
285+
"to": "now"
286+
},
287+
"timepicker": {},
288+
"timezone": "browser",
289+
"title": "GreptimeDB Logs",
290+
"uid": "edx5veo4rd3wge2",
291+
"version": 1
292+
}

‎grafana/dashboards/cluster/dashboard.json renamed to ‎grafana/dashboards/metrics/cluster/dashboard.json

File renamed without changes.

‎grafana/dashboards/cluster/dashboard.md renamed to ‎grafana/dashboards/metrics/cluster/dashboard.md

File renamed without changes.

‎grafana/dashboards/cluster/dashboard.yaml renamed to ‎grafana/dashboards/metrics/cluster/dashboard.yaml

File renamed without changes.

‎grafana/dashboards/standalone/dashboard.json renamed to ‎grafana/dashboards/metrics/standalone/dashboard.json

File renamed without changes.

‎grafana/dashboards/standalone/dashboard.md renamed to ‎grafana/dashboards/metrics/standalone/dashboard.md

File renamed without changes.

‎grafana/dashboards/standalone/dashboard.yaml renamed to ‎grafana/dashboards/metrics/standalone/dashboard.yaml

File renamed without changes.

0 commit comments

Comments
 (0)