katacoda: Small cleanup and added playground demo. (thanos-io#3463)

Signed-off-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: Oghenebrume50 <[email protected]>

Author: bwplotka

tutorials/katacoda/thanos-pathway.json

@@ -1,6 +1,6 @@
 {
   "title": "Learn Thanos",
-  "description": "Introduction, Tips and Advanced Tutorials for Thanos: the CNCF, Global, Scalable System for Prometheus Metrics with cheap, Long-term Storage Capabilities. Course version: v0.2",
+  "description": "Introduction, Tips and Advanced Tutorials for Thanos: the CNCF, Global, Scalable System for Prometheus Metrics with cheap, Long-term Storage Capabilities. Course version: v0.3",
   "icon": "fa-thanos",
   "courses": [
     {
@@ -14,29 +14,19 @@
       "description": "Learn how to extend your metric retention in a cheap and easy way with Thanos."
     },
     {
-      "course_id": "3-meta-monitoring",
-      "title": "Intro: Global and meta alerts with Thanos",
-      "description": "In progress. Stay Tuned!"
-    },
-    {
-      "course_id": "4-cross-cluster-comm",
-      "title": "Advanced: Connecting remote Prometheuses to Thanos using simple Envoy setup",
-      "description": "In progress. Stay Tuned!"
-    },
-    {
-      "course_id": "5-remote-receiver",
-      "title": "Advanced: Using Prometheus remote write to stream metrics to Thanos",
-      "description": "In progress. Stay Tuned!"
+      "course_id": "7-multi-tenancy",
+      "title": "Advanced: Achieving Multi-Tenancy with Thanos",
+      "description": "Extend your Prometheus setup for Multiple Teams use with Thanos"
     },
     {
-      "course_id": "6-caching",
-      "title": "Advanced: Query low tail latency with low cost: Introducing caching to Thanos",
-      "description": "In progress. Stay Tuned!"
+      "course_id": "x-playground",
+      "title": "Playground: All-in, Interactive Thanos un-guided Demo",
+      "description": "Demo playground that spins all the components from previous courser together."
     },
     {
-      "course_id": "7-multi-tenancy",
-      "title": "Advanced: Achieving Multi-Tenancy with Thanos",
-      "description": "Extend your Prometheus setup for Multiple Teams use with Thanos"
+      "course_id": "x-more-to-come",
+      "title": "<Your Tutorial>",
+      "description": "Anything you want to like to see? Or you want to help? Let us know!"
     }
   ]
 }

tutorials/katacoda/thanos/2-lts/step1.md

@@ -25,11 +25,15 @@ mkdir -p /root/prom-eu1 && docker run -i quay.io/thanos/thanosbench:v0.2.0-rc.1
 On successful block creation you should see following log lines:
 
 ```
-level=info ts=2020-10-20T18:28:42.625041939Z caller=block.go:87 msg="all blocks done" count=X
+level=info ts=2020-10-20T18:28:42.625041939Z caller=block.go:87 msg="all blocks done" count=13
 level=info ts=2020-10-20T18:28:42.625100758Z caller=main.go:118 msg=exiting cmd="block gen"
 ```
 
-Run `ls -lR /root/prom-eu1` to see dozens of generated TSDB blocks.
+Run below command to see dozens of generated TSDB blocks:
+
+```
+ls -lR /root/prom-eu1
+```{{execute}}
 
 ## Prometheus Configuration File
 
@@ -103,7 +107,7 @@ docker run -d --net=host --rm \
 
 Once started you should be able to reach the Prometheus instance here and query.. 1 year of data!
 
-* [Prometheus-0 EU1](https://[[HOST_SUBDOMAIN]]-9090-[[KATACODA_HOST]].environments.katacoda.com/)
+* [Prometheus-0 EU1](https://[[HOST_SUBDOMAIN]]-9090-[[KATACODA_HOST]].environments.katacoda.com/graph?g0.range_input=1y&g0.expr=continuous_app_metric0&g0.tab=0)
 
 ## Thanos Sidecar & Querier
 
@@ -140,6 +144,6 @@ Similar to previous course let's check if the Querier works as intended. Let's l
 
 This should list the sidecar, including the external labels.
 
-On graph you should also see our 5 series for 1y time, thanks to Prometheus and sidecar StorAPI: [Graph](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com).
+On graph you should also see our 5 series for 1y time, thanks to Prometheus and sidecar StorAPI: [Graph](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/https://2886795307-9091-ollie02.environments.katacoda.com/graph?g0.range_input=1y&g0.max_source_resolution=0s&g0.expr=continuous_app_metric0&g0.tab=0).
 
 Click `Continue` to see how we can move this data to much cheaper and easier to operate object storage.

tutorials/katacoda/thanos/2-lts/step3.md

@@ -41,19 +41,49 @@ docker run -d --net=host --rm \
 
 ## How to query Thanos store data?
 
-In this step, we will see how we can query Thanos store data which has access to historical data from the `thanos` bucket, and let's play with this setup a bit.
+In this step, we will see how we can query Thanos store data which has access to historical data from the `thanos` bucket, and play with this setup a bit.
 
-Click on the [Querier UI `Graph` page](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/graph) and try querying data for a year or two by inserting metrics [continuous_app_metric0](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/graph?g0.range_input=1y&g0.max_source_resolution=0s&g0.expr=continuous_app_metric0&g0.tab=0). Make sure `deduplication` is selected and you will be able to discover all the data fetched by Thanos store.
+Currently querier does not know about store yet. Let's change it by adding Store Gateway gRPC address `--store 127.0.0.1:19191` to Querier:
 
-![](https://github.com/soniasingla/thanos/raw/master/tutorials/katacoda/thanos/2-lts/query.png)
+```
+docker stop querier && \
+docker run -d --net=host --rm \
+   --name querier \
+   quay.io/thanos/thanos:v0.16.0 \
+   query \
+   --http-address 0.0.0.0:9091 \
+   --query.replica-label replica \
+   --store 127.0.0.1:19190 \
+   --store 127.0.0.1:19191
+```{{execute}}
+
+Click on the Querier UI `Graph` page and try querying data for a year or two by inserting metrics `continuous_app_metric0` ([Query UI](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/graph?g0.range_input=1y&g0.max_source_resolution=0s&g0.expr=continuous_app_metric0&g0.tab=0)). Make sure `deduplication` is selected and you will be able to discover all the data fetched by Thanos store.
+
+![](https://github.com/thanos-io/thanos/raw/master/tutorials/katacoda/thanos/2-lts/query.png)
 
 Also, you can check all the active endpoints located by thanos-store by clicking on [Stores](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/stores).
 
-We've added Thanos Query, a web and API frontend that can query a Prometheus instance and Thanos Store at the same time, which gives transparent access to the archived blocks and real-time metrics. The vanilla PromQL Prometheus engine used for evaluating the query deduces what time series and for what time ranges we need to fetch the data. Also, StoreAPIs propagate external labels and the time range they have data for, so we can do basic filtering on this. However, if you don't specify any of these in the query (only "up" series) the querier concurrently asks all the StoreAPI servers. It might cause a duplication of results between sidecar and store data.
+### What we know so far?
+
+We've added Thanos Query, a component that can query a Prometheus instance and Thanos Store at the same time,
+which gives transparent access to the archived blocks and real-time metrics. The vanilla PromQL Prometheus engine used inside Query deduces
+what time series and for what time ranges we need to fetch the data for.
+
+Also, StoreAPIs propagate external labels and the time range they have data for, so we can do basic filtering on this.
+However, if you don't specify any of these in the query (only "up" series) the querier concurrently asks all the StoreAPI servers.
+
+The potential duplication of data between Prometheus+sidecar results and store Gateway will be transparently handled and invisible in results.
+
+### Checking what StoreAPIs are involved in query
+
+Another interesting question here is how to ensure if we query the data from bucket only?
+
+We can check this by visitng the [New UI]((https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/new/graph?g0.expr=&g0.tab=0&g0.stacked=0&g0.range_input=1h&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=[])), inserting `continuous_app_metric0` metrics again with 1 year time range of graph,
+and click on `Enable Store Filtering`.
 
-Now, another interesting question here is how to ensure if we query the data from bucket only?
+This allows us to filter stores and helps in debugging from where we are querying the data exactly.
 
-We can check this by visitng the [New UI]((https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/new/graph?g0.expr=&g0.tab=0&g0.stacked=0&g0.range_input=1h&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=[])), inserting `continuous_app_metric0` metrics again with 1 year time range of graph, and click on `Enable Store Filtering`. This allows us to filter stores and helps in debugging from where we are querying the data exactly.
+Let's chose only `127.0.0.1:19191`, so store gateway. This query will only hit that store to retrieve data, so we are sure that store works.
 
 ## Question Time? 🤔
 

tutorials/katacoda/thanos/2-lts/step4.md

@@ -1,18 +1,21 @@
 # Step 4 - Thanos Compactor
 
-In this step, we will install Thanos Compactor which applies the compaction procedure of the Prometheus 2.0 storage engine to block data in object storage.
+In this step, we will install Thanos Compactor which applies the compaction, retention, deletion and downsampling operations
+on the TSDB block data object storage.
 
 Before, moving forward, let's take a closer look at what the `Compactor` component does:
 
+> Note: Thanos Compactor is currently designed to be run as a singleton. Unavailability (hours) is not problematic as it does not serve any live requests.
 ## Compactor
 
-The `Compactor` is an essential component that operates on a single object storage bucket to compact, down-sample, apply retention, to the TSDB blocks held inside, thus, making queries on historical data more efficient. It creates aggregates of old metrics (based upon the rules).
+The `Compactor` is an essential component that operates on a single object storage bucket to compact, downsample, apply retention, to the TSDB blocks held inside,
+thus, making queries on historical data more efficient. It creates aggregates of old metrics (based upon the rules).
 
 It is also responsible for downsampling of data, performing 5m downsampling after 40 hours, and 1h downsampling after 10 days.
 
 If you want to know more about Thanos Compactor, jump [here](https://thanos.io/tip/components/compact.md/).
 
-**Note**: Thanos Compactor is mandatory if you use object storage otherwise Thanos Store Gateway will be too slow without using a compactor.
+> Note: Thanos Compactor is mandatory if you use object storage otherwise Thanos Store Gateway will be too slow without using a compactor.
 
 ## Deploying Thanos Compactor
 
@@ -34,15 +37,20 @@ The flag `wait` is used to make sure all compactions have been processed while `
 
 ## Setup Verification
 
-To check if compactor works fine, we can look at the [Bucket View](https://[[HOST_SUBDOMAIN]]-19095-[[KATACODA_HOST]].environments.katacoda.com/loaded).
+To check if compactor works fine, we can look at the [Bucket View](https://[[HOST_SUBDOMAIN]]-19095-[[KATACODA_HOST]].environments.katacoda.com/new/loaded).
 
 Now, if we click on the blocks, they will provide us all the metadata (Series, Samples, Resolution, Chunks, and many more things).
 
 ## Compaction and Downsampling
 
-When we query large historical data there will be definitely many complexities making the queries slower and slower as we retrieve year's worth of data. Thus, Thanos uses the technique called downsampling (a process of reducing the sampling rate of the signal) to keep the queries responsive, and no special configuration is required to perform this process. The Compactor applies compaction to the bucket data and also completes the downsampling for historical data.
+When we query large historical data there are millions of samples that we need to go through which makes the queries slower and slower as we retrieve year's worth of data.
 
-Now, click on the [Querier](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/new/graph?g0.expr=&g0.tab=0&g0.stacked=0&g0.range_input=1h&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=[]) and insert metrics `continuous_app_metric0` with 1 year time range of graph, and also, click on `Enable Store Filtering`.
+Thus, Thanos uses the technique called downsampling (a process of reducing the sampling rate of the signal) to keep the queries responsive,
+and no special configuration is required to perform this process.
+
+The Compactor applies compaction to the bucket data and also completes the downsampling for historical data.
+
+To expierience this, click on the [Querier](https://[[HOST_SUBDOMAIN]]-9091-[[KATACODA_HOST]].environments.katacoda.com/new/graph?g0.expr=&g0.tab=0&g0.stacked=0&g0.range_input=1h&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=[]) and insert metrics `continuous_app_metric0` with 1 year time range of graph, and also, click on `Enable Store Filtering`.
 
 Let's try querying `Max 5m downsampling` data, it uses 5m resolution and it will be faster than the raw data. Also, Downsampling is built on top of data, and never done on **young** data.
 

tutorials/katacoda/thanos/3-meta-monitoring/intro.md

@@ -1,6 +1,6 @@
 {
-  "title": "Intro: Global and meta alerts with Thanos Ruler",
-  "description": "In progress. Stay Tuned!",
+  "title": "<Your Tutorial>",
+  "description": "Anything you want to like to see? Or you want to help? Let us know!",
   "details": {
     "steps": [
     ],

tutorials/katacoda/thanos/4-cross-cluster-comm/courseBase.sh

@@ -1,6 +1,6 @@
-# In progress
+# Work in Progress
 
-🚧 This tutorial is in progress. 🚧
+🚧 More tutorials are in progress. 🚧
 
 Do you want to see this soon or you want to help us?
 

tutorials/katacoda/thanos/4-cross-cluster-comm/index.json

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+docker pull quay.io/prometheus/prometheus:v2.20.0
+docker pull quay.io/thanos/thanos:v0.16.0
+docker pull quay.io/thanos/thanosbench:v0.2.0-rc.1
+docker pull minio/minio:RELEASE.2019-01-31T00-31-19Z
+
+mkdir /root/editor

tutorials/katacoda/thanos/5-remote-receiver/courseBase.sh

@@ -0,0 +1,25 @@
+# Summary
+
+Uff done.
+
+<img src="https://i.imgflip.com/4mw3l5.jpg" title="yoda"/>
+
+### Cleanup if you ran locally
+
+```
+docker stop prom-eu1-0
+docker stop prom-eu1-1
+docker stop prom-us1-0
+docker stop prom-eu1-0-sidecar
+docker stop prom-eu1-1-sidecar
+docker stop prom-us1-0-sidecar
+docker stop querier
+docker stop minio
+docker stop store-gateway
+docker stop compactor
+```
+
+### Feedback
+
+Do you see any bug, typo in the tutorial or you have some feedback for us?
+Let us know on https://github.com/thanos-io/thanos or #thanos slack channel linked on https://thanos.io

tutorials/katacoda/thanos/5-remote-receiver/index.json

@@ -0,0 +1,30 @@
+{
+  "title": "Playground: All-in, Interactive Thanos un-guided Demo",
+  "description": "\uD83E\uDD14\uD83E\uDD14",
+  "difficulty": "YOLO",
+  "details": {
+    "steps": [
+      {
+        "title": "Yolo Demo Resources",
+        "text": "step1.md"
+      },
+      {
+        "title": "Yolo Demo Resources",
+        "text": "step2.md"
+      }
+    ],
+    "intro": {
+      "text": "intro.md",
+      "courseData": "courseBase.sh"
+    }
+  },
+  "environment": {
+    "uilayout": "terminal",
+    "uisettings": "yaml",
+    "showdashboard": true,
+    "uieditorpath": "/root/editor"
+  },
+  "backend": {
+    "imageid": "docker-direct"
+  }
+}

tutorials/katacoda/thanos/5-remote-receiver/intro.md

@@ -0,0 +1,18 @@
+# Demo Resources
+
+Let's go!
+
+This playground was created for `@rawkode` live demo made by `@bwplotka` and David McKay.
+
+You can watch all here: https://www.youtube.com/watch?v=j4TAGO019HU&feature=emb_title&ab_channel=DavidMcKay
+
+Otherwise, have fun with those resources!
+
+### Feedback
+
+Do you see any bug, typo in the tutorial or you have some feedback for us?
+Let us know on https://github.com/thanos-io/thanos or #thanos slack channel linked on https://thanos.io
+
+### Contributed by:
+
+* Bartek [@bwplotka](https://bwplotka.dev/)