Skip to content

Commit 8ae3ce1

Browse files
committed
update
1 parent 23a956d commit 8ae3ce1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+202
-1549
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,4 @@ data/
1717

1818
/.idea
1919
/*.iml
20+
website/public/

docs/components/bucket.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ It is normally run as a stand alone command to aid with troubleshooting.
1111

1212
Example:
1313

14-
```
14+
```bash
1515
$ thanos bucket verify --objstore.config-file=bucket.yml
1616
```
1717

docs/components/compact.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ It is generally not semantically concurrency safe and must be deployed as a sing
1111

1212
Example:
1313

14-
```
14+
```bash
1515
$ thanos compact --data-dir /tmp/thanos-compact --objstore.config-file=bucket.yml
1616
```
1717

docs/components/query.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ on how to connect querier with desired StoreAPIs.
1313

1414
Querier currently is fully stateless and horizontally scalable.
1515

16-
```
16+
```bash
1717
$ thanos query \
1818
--http-address "0.0.0.0:9090" \
1919
--store "<store-api>:<grpc-port>" \

docs/components/rule.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ The rule component evaluates Prometheus recording and alerting rules against ran
1414

1515
The data of each rule node can be labeled to satisfy the clusters labeling scheme. High-availability pairs can be run in parallel and should be distinguished by the designated replica label, just like regular Prometheus servers.
1616

17-
```
17+
```bash
1818
$ thanos rule \
1919
--data-dir "/path/to/data" \
2020
--eval-interval "30s" \

docs/components/sidecar.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,14 @@ Prometheus servers connected to the Thanos cluster via the sidecar are subject t
1919
The retention is recommended to not be lower than three times the block duration. This achieves resilience in the face of connectivity issues
2020
to the object storage since all local data will remain available within the Thanos cluster. If connectivity gets restored the backlog of blocks gets uploaded to the object storage.
2121

22-
```console
22+
```bash
2323
$ prometheus \
2424
--storage.tsdb.max-block-duration=2h \
2525
--storage.tsdb.min-block-duration=2h \
2626
--web.enable-lifecycle
2727
```
2828

29-
```console
29+
```bash
3030
$ thanos sidecar \
3131
--tsdb.path "/path/to/prometheus/data/dir" \
3232
--prometheus.url "http://localhost:9090" \

docs/components/store.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ menu: "components"
99
The store component of Thanos implements the Store API on top of historical data in an object storage bucket. It acts primarily as an API gateway and therefore does not need significant amounts of local disk space. It joins a Thanos cluster on startup and advertises the data it can access.
1010
It keeps a small amount of information about all remote blocks on local disk and keeps it in sync with the bucket. This data is generally safe to delete across restarts at the cost of increased startup times.
1111

12-
```
12+
```bash
1313
$ thanos store \
1414
--data-dir "/local/state/data/dir" \
1515
--cluster.peers "thanos-cluster.example.org" \

docs/design.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ The sidecar implements the gRPC service on top of Prometheus' [HTTP and remote-r
3030

3131
Data sources that persist their data for long-term storage do so via the Prometheus 2.0 storage engine. The storage engine periodically produces immutable blocks of data for a fixed time range. A block is a directory with a handful of larger files containing all sample data and peristed indices that are required to retrieve the data:
3232

33-
```
33+
```bash
3434
01BX6V6TY06G5MFQ0GPH7EMXRH
3535
├── chunks
3636
│ ├── 000001

docs/getting-started.md

+18-13
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
title: Getting Started
33
type: docs
44
menu: "thanos"
5+
weight: 1
56
---
67

78
# Getting started
@@ -11,6 +12,10 @@ Thanos provides a global query view, data backup, and historical data access as
1112
In this quick-start guide, we will configure Thanos and all components mentioned to work against a Google Cloud Storage bucket.
1213
At the moment, Thanos is able to use [different storage providers](storage.md), with the ability to add more providers as necessary.
1314

15+
## Architecture Overview
16+
17+
![architecture_overview](../img/arch.jpg)
18+
1419
## Requirements
1520

1621
* One or more [Prometheus](https://prometheus.io) v2.2.1+ installations
@@ -24,7 +29,7 @@ You can find the latest Thanos release [here](https://github.com/improbable-eng/
2429
If you want to build Thanos from source -
2530
with a working installation of the Go [toolchain](https://github.com/golang/tools) (`GOPATH`, `PATH=${GOPATH}/bin:${PATH}`), Thanos can be downloaded and built by running:
2631

27-
```
32+
```bash
2833
go get -d github.com/improbable-eng/thanos/...
2934
cd ${GOPATH}/src/github.com/improbable-eng/thanos
3035
make
@@ -52,7 +57,7 @@ NOTE: If you want to use `reload.*` flags for sidecar, make sure you enable `rel
5257

5358
The following configures the sidecar to only backup Prometheus data into a chosen object storage:
5459

55-
```
60+
```bash
5661
thanos sidecar \
5762
--tsdb.path /var/prometheus \ # TSDB data directory of Prometheus
5863
--prometheus.url "http://localhost:9090" \
@@ -75,7 +80,7 @@ The Sidecar component implements and exposes gRPC _[Store API](/pkg/store/storep
7580

7681
Let's extend the Sidecar in the previous section to connect to a Prometheus server, and expose the Store API.
7782

78-
```
83+
```bash
7984
thanos sidecar \
8085
--tsdb.path /var/prometheus \
8186
--objstore.config-file bucket_config.yaml \ # Bucket config file to send data to
@@ -93,7 +98,7 @@ Prometheus allows the configuration of "external labels" of a given Prometheus i
9398

9499
Every Prometheus instance must have a globally unique set of identifying labels. For example, in Prometheus's configuration file:
95100

96-
```
101+
```yaml
97102
global:
98103
external_labels:
99104
region: eu-west
@@ -112,7 +117,7 @@ Query also implements Prometheus's offical HTTP API and can thus be used with ex
112117

113118
Below, we will set up a Query to connect to our Sidecars, and expose its HTTP UI.
114119

115-
```
120+
```bash
116121
thanos query \
117122
--http-address 0.0.0.0:19192 \ # HTTP Endpoint for Query UI
118123
--store 1.2.3.4:19090 \ # Static gRPC Store API Address for the query node to query
@@ -127,7 +132,7 @@ The Query component is also capable of deduplicating data collected from Prometh
127132

128133
A typical choice is simply the label name "replica" while letting the value be whatever you wish. For example, you might set up the following in Prometheus's configuration file:
129134

130-
```
135+
```yaml
131136
global:
132137
external_labels:
133138
region: eu-west
@@ -138,7 +143,7 @@ global:
138143

139144
Reload your Prometheus instances, and then, in Query, we will enable `replica` as the label we want to enable deduplication to occur on:
140145

141-
```
146+
```bash
142147
thanos query \
143148
--http-address 0.0.0.0:19192 \
144149
--store 1.2.3.4:19090 \
@@ -158,7 +163,7 @@ The metadata includes the information about time windows and external labels for
158163
There are various ways to tell query component about the StoreAPIs it should query data from. The simplest way is to use a static list of well known addresses to query.
159164
These are repeatable so can add as many endpoint as needed. You can put DNS domain prefixed by `dns://` or `dns+srv://` to have Thanos Query do an `A` or `SRV` lookup to get all required IPs to communicate with.
160165

161-
```
166+
```bash
162167
thanos query \
163168
--http-address 0.0.0.0:19192 \ # Endpoint for Query UI
164169
--grpc-address 0.0.0.0:19092 \ # gRPC endpoint for Store API
@@ -175,7 +180,7 @@ Given a sidecar we can have it join a gossip cluster by advertising itself to ot
175180

176181
NOTE: Gossip will be removed. See [here](/docs/proposals/approved/201809_gossip-removal.md) why. New FileSD with DNS support is enabled and described [here](/docs/service_discovery.md)
177182

178-
```
183+
```bash
179184
thanos sidecar \
180185
--prometheus.url http://localhost:9090 \
181186
--tsdb.path /var/prometheus \
@@ -193,7 +198,7 @@ When a peer advertises itself / joins a gossip cluster it sends information abou
193198

194199
Once the Peer joins the cluster it will periodically update the information it sends out with new / updated information about other peers and the time windows for the metrics that it can access.
195200

196-
```
201+
```bash
197202
thanos query \
198203
--http-address 0.0.0.0:19192 \ # Endpoint for Query UI
199204
--grpc-address 0.0.0.0:19092 \ # gRPC endpoint for Store API
@@ -204,7 +209,7 @@ thanos query \
204209

205210
You can mix both static `store` and `cluster` based approaches:
206211

207-
```
212+
```bash
208213
thanos query \
209214
--http-address 0.0.0.0:19192 \ # Endpoint for Query UI
210215
--grpc-address 0.0.0.0:19092 \ # gRPC endpoint for Store API
@@ -233,7 +238,7 @@ As the sidecar backs up data into the object storage of your choice, you can dec
233238
The store gateway does just that by implementing the same gRPC data API as the sidecars but backing it with data it can find in your object storage bucket.
234239
Just like sidecars and query nodes, the store gateway exposes StoreAPI and needs to be discovered by Thanos Querier.
235240

236-
```
241+
```bash
237242
thanos store \
238243
--data-dir /var/thanos/store \ # Disk space for local caches
239244
--objstore.config-file bucket_config.yaml \ # Bucket to fetch data from
@@ -251,7 +256,7 @@ A local Prometheus installation periodically compacts older data to improve quer
251256

252257
The compactor component simple scans the object storage and processes compaction where required. At the same time it is responsible for creating downsampled copies of data to speed up queries.
253258

254-
```
259+
```bash
255260
thanos compact \
256261
--data-dir /var/thanos/compact \ # Temporary workspace for data processing
257262
--objstore.config-file bucket_config.yaml \ # Bucket where to apply the compacting

docs/service-discovery.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,10 @@
1-
# Thanos Service Discovery
1+
---
2+
title: Service Discovery
3+
type: docs
4+
menu: "thanos"
5+
---
6+
7+
# Service Discovery
28

39
Service discovery has a vital place in Thanos components. It allows Thanos to discover different set API targets required to perform certain operations.
410
This logic is meant to replace Gossip that [is planned to be removed.](/docs/proposals/approved/201809_gossip-removal.md)

docs/storage.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
title: Object Storage
33
type: docs
44
menu: "thanos"
5+
markup: mmark
56
---
67

78
# Object Storage
@@ -14,6 +15,7 @@ All clients are configured using `--objstore.config-file` to reference to the co
1415

1516
Current object storage client implementations:
1617

18+
{.table .table-striped}
1719
| Provider | Maturity | Auto-tested on CI | Maintainers |
1820
|----------------------|-------------------|-----------|---------------|
1921
| Google Cloud Storage | Stable (production usage) | yes | @bwplotka |

website/config.toml

+6-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
baseURL = "https://adrien-f.github.io/thanos/"
22
languageCode = "en-us"
33
title = "Thanos - Highly available Prometheus setup with long term storage capabilities"
4+
pygmentsCodeFences = true
5+
pygmentsUseClasses = true
46

5-
[permalinks]
6-
docs = "/docs/:filename/"
7+
[params]
8+
SlackInvite = "https://join.slack.com/t/improbable-eng/shared_invite/enQtMzQ1ODcyMzQ5MjM4LWY5ZWZmNGM2ODc5MmViNmQ3ZTA3ZTY3NzQwOTBlMTkzZmIxZTIxODk0OWU3YjZhNWVlNDU3MDlkZGViZjhkMjc"
9+
GithubUser = "improbable-eng"
10+
GithubProject = "thanos"

website/layouts/_default/baseof.html

+15-24
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,14 @@
88
crossorigin="anonymous">
99
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.7.1/css/all.css" integrity="sha384-fnmOCqbTlWIlj8LyTjo7mOUStjsKC4pOpQbqyi7RrhN7udi9RwhKkMHpvLbHG9Sr"
1010
crossorigin="anonymous">
11-
<link rel="stylesheet" href="{{ "/main.css" | relURL }}">
11+
<link rel="stylesheet" href="{{ "/main.css" | relURL }}" />
12+
<link rel="stylesheet" href="{{ "/syntax.css" | relURL }}" />
13+
<link rel="icon" href="{{ "/icon-light.png" | relURL }}" />
1214
<title>{{ block "title" . }}{{ .Site.Title }}{{ end }}</title>
15+
<meta property="og:image" content="{{ "/icon-light.png" | relURL }}" />
16+
<meta property="og:type" content="object" />
17+
<meta property="og:title" content="Thanos" />
18+
<meta property="og:description" content="{{ .Site.Title }}" />
1319
</head>
1420

1521
<body>
@@ -22,13 +28,13 @@
2228
<div class="collapse navbar-collapse" id="navbarSupportedContent">
2329
<ul class="navbar-nav ml-auto">
2430
<li class="nav-item">
25-
<a class="nav-link" href="{{ "/docs/getting-started" | relURL }}">Docs</a>
31+
<a class="nav-link" href="{{ "/docs/getting-started" | relURL }}">Documentation</a>
2632
</li>
2733
<li class="nav-item">
28-
<a class="nav-link" href="#">Download</a>
34+
<a class="nav-link" href="https://github.com/{{ .Site.Params.GithubUser }}/{{ .Site.Params.GithubProject }}/releases">Download</a>
2935
</li>
3036
<li class="nav-item">
31-
<a href="#" class="nav-link"><i class="fab fa-github"></i> GitHub</a>
37+
<a href="https://github.com/{{ .Site.Params.GithubUser }}/{{ .Site.Params.GithubProject }}" class="nav-link"><i class="fab fa-fw fa-github"></i> GitHub</a>
3238
</li>
3339
</ul>
3440
</div>
@@ -38,26 +44,12 @@
3844
<div class="container">
3945
<div class="row py-5">
4046
<div class="col">
41-
<h6 class="font-weight-bold">Documentation</h6>
47+
<h6 class="font-weight-bold">Thanos</h6>
4248
<ul class="list-unstyled">
43-
<li><a href="#" class="text-reset">Lorem ipsum dolor sit amet</a></li>
44-
<li><a href="#" class="text-reset">Consectetur adipiscing elit</a></li>
45-
</ul>
46-
</div>
47-
<div class="col">
48-
<h6 class="font-weight-bold">Community</h6>
49-
<ul class="list-unstyled">
50-
<li><a href="#" class="text-reset">Slack</a></li>
51-
<li><a href="#" class="text-reset">GitHub</a></li>
52-
<li><a href="#" class="text-reset">User Showcase</a></li>
53-
</ul>
54-
</div>
55-
<div class="col">
56-
<h6 class="font-weight-bold">More</h6>
57-
<ul class="list-unstyled">
58-
<li><a href="#" class="text-reset">Slack</a></li>
59-
<li><a href="#" class="text-reset">Github</a></li>
60-
<li><a href="#" class="text-reset">User Showcase</a></li>
49+
<li><a href="{{ "/docs/getting-started" | relURL }}" class="text-reset">Documentation</a></li>
50+
<li><a href="{{ .Site.Params.SlackInvite }}" class="text-reset">Slack</a></li>
51+
<li><a href="https://github.com/{{ .Site.Params.GithubUser }}/{{ .Site.Params.GithubProject }}" class="text-reset">GitHub</a></li>
52+
<li><a href="https://github.com/{{ .Site.Params.GithubUser }}/{{ .Site.Params.GithubProject }}/issues" class="text-reset">Issues</a></li>
6153
</ul>
6254
</div>
6355
</div>
@@ -70,5 +62,4 @@ <h6 class="font-weight-bold">More</h6>
7062
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/js/bootstrap.min.js" integrity="sha384-B0UglyR+jN6CkvvICOB2joaf5I4l3gm9GU6Hc1og6Ls7i6U/mkkaduKaBhlAXv9k"
7163
crossorigin="anonymous"></script>
7264
</body>
73-
7465
</html>

website/layouts/component/single.html

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{{ define "main" }}
2+
<div class="container">
3+
<div class="row py-5">
4+
<div class="col col-3">
5+
{{ $currentPage := . }}
6+
<h5>Thanos</h5>
7+
<ul class="list-unstyled">
8+
{{ range .Site.Menus.thanos }}
9+
<li class="ml-3">
10+
<a href="{{ .URL }}">{{ .Title }}</a>
11+
</li>
12+
{{ end }}
13+
</ul>
14+
<h5 class="mt-3">Components</h5>
15+
<ul class="list-unstyled">
16+
{{ range .Site.Menus.components }}
17+
<li class="ml-3">
18+
<a href="{{ .URL }}">{{ .Title }}</a>
19+
</li>
20+
{{ end }}
21+
</ul>
22+
</div>
23+
<div class="col col-9">
24+
{{ .Content }}
25+
</div>
26+
</div>
27+
</div>
28+
{{ end }}

0 commit comments

Comments
 (0)