Export or dump data?

[mr-rulez]

First of all, TickTock DB is running super smooth on my Raspi 3b+! Many thanks!!!

Now, I have some more questions:

1. How would you dump or export data for one or all metrics?
2. If the same data point (same timestamp, same metric-ID, same value, etc.) is put / inserted once again, will it be overwritten or result into two entries?
3. Is there an open source data browser graphical user interface positively tested working with Ticktock DB?

[ylin30]

First of all, TickTock DB is running super smooth on my Raspi 3b+! Many thanks!!!

Nice! Frankly, we don't have RPI 3b+ for testing yet.

Now, I have some more questions:

1. How would you dump or export data for one or all metrics?

# TickTock home directory.
; tsdb.data.dir = <cwd>/data

We don't provide tools to dump specific metrics. But all metrics' data are in tsdb.data.dir (default path: /data). You can just copy all the files into another dir, and start a new TickTockDB with tsdb.data.dir pointing to the data dir.

There is binary, bin/inspect, which can read data in data dir. You may give a try. It is a bit slow since we just use it as an internal debug tool so not optimization efforts yet.

2. If the same data point (same timestamp, same metric-ID, same value, etc.) is put / inserted once again, will it be overwritten or result into two entries?

Overwritten.

3. Is there an open source data browser graphical user interface positively tested working with Ticktock DB?

We only use Grafana. Isn't it good enough? Please see user guide.

Thanks,
Yi

[mr-rulez]

I tried inspect tool. With paramaters "-v -d <data_dir>" it results in listing all timestamps and their values, but I did not find clear reference to which of the metric-IDs they are assigned and I am missing labels at all.
How would you dump all rawxmetric data points of all datasets to a human readable format?
The documented queries are averaging values for many metric-IDs per a specific timestamp, which often does not make sense.

There must be a way as Grafana is also querying the raw results :)

And once more many thanks for this cool lightweight TSDB!!!

[ylin30]

I tried inspect tool. With paramaters "-v -d <data_dir>" it results in listing all timestamps and their values, but I did not find clear reference to which of the metric-IDs they are assigned and I am missing labels at all. How would you dump all rawxmetric data points of all datasets to a human readable format? The documented queries are averaging values for many metric-IDs per a specific timestamp, which often does not make sense.

There must be a way as Grafana is also querying the raw results :)

Oh, I see what you want. Yes, Grafana can send query with "disable downsample" to get all raw data points, so can we using curl.
Grafana uses Json format. You can either use the same payload (If you are familiar with browser debug, you can find it in browser's inspect/network/query...) Please look at

You can also use the concise form of Opentsdb query. Note that in m=sum:1m-avg:cpu.idle,

• sum: The sum of all time series matching this query at each timestamp.
• 1m-avg: For each time series, do downsampling by avg in 1 mintue.

m=sum:cpu.idle means downsampling is disabled (just returned raw data, but still do sum of all time series). Let me give you another concrete example.

[Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=sum:cpu.usr\{host=rpi4,cpu=*\}'
[{"metric":"cpu.usr","tags":{"cpu":"0","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":2.6000000000000103,"1678258524":0.1400000000000152,"1678258539":0.0000000000000155,"1678258554":0.270000000000015}},{"metric":"cpu.usr","tags":{"cpu":"1","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":7.3500000000000458,"1678258524":6.4700000000000477,"1678258539":1.0700000000000589,"1678258554":0.8000000000000594}},{"metric":"cpu.usr","tags":{"cpu":"2","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":13.0999999999999783,"1678258524":7.6199999999999894,"1678258539":0.6000000000000041,"1678258554":0.0000000000000053}},{"metric":"cpu.usr","tags":{"cpu":"3","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":1.210000000000008,"1678258524":3.5800000000000032,"1678258539":0.2000000000000099,"1678258554":0.0700000000000102}}]

Note I use sum:cpu.usr and a wild cast tag cpu=* above. So all the raw data of 4 time series are returned.

[Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=sum:1m-avg:cpu.usr\{host=rpi4,cpu=*\}'
[{"metric":"cpu.usr","tags":{"cpu":"0","host":"rpi4"},"aggregateTags":[],"dps":{"1678258500":0.752500000000014}},{"metric":"cpu.usr","tags":{"cpu":"1","host":"rpi4"},"aggregateTags":[],"dps":{"1678258500":3.9225000000000527}},{"metric":"cpu.usr","tags":{"cpu":"2","host":"rpi4"},"aggregateTags":[],"dps":{"1678258500":5.3299999999999947}},{"metric":"cpu.usr","tags":{"cpu":"3","host":"rpi4"},"aggregateTags":[],"dps":{"1678258500":1.2650000000000079}}][Yi-MBP ticktock (timestamp)]$

Note that I use sum:1m-avg:cpu.usr and cpu=*. It still returns 4 time series' data. But each time series has only 1 data point which is round to minute.

[Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=sum:cpu.usr\{host=rpi4,cpu=0\}'
[{"metric":"cpu.usr","tags":{"cpu":"0","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":2.6000000000000103,"1678258524":0.1400000000000152,"1678258539":0.0000000000000155,"1678258554":0.270000000000015}}][Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=sum:cpu.usr\{host=rpi4,cpu=0\}'
[Yi-MBP ticktock (timestamp)]$
[Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=avg:cpu.usr\{host=rpi4,cpu=0\}'
[{"metric":"cpu.usr","tags":{"cpu":"0","host":"rpi4"},"aggregateTags":[],"dps":{"1678258509":2.6000000000000103,"1678258524":0.1400000000000152,"1678258539":0.0000000000000155,"1678258554":0.270000000000015}}][Yi-MBP ticktock (timestamp)]$

Note that I use cpu=0 above. Then sum or avg do not matter.

[ylin30]

[Yi-MBP ticktock (timestamp)]$ curl 'http://localhost:6182/api/query?start=1678258500&end=1678258560&m=sum:1m-avg:cpu.usr\{host=rpi4\}'
[{"metric":"cpu.usr","tags":{"host":"rpi4"},"aggregateTags":["cpu"],"dps":{"1678258500":11.2700000000000689}}]

A further example, if you get rid of cpu=*, then all data points of the 4 time series (cpu=0,1,2,3) will be sum up as one data point.