While running stress test I am getting err="the StoreAPI responded with zero metric names"

[dmilind]

I am playing with thanosbench to perform a stress test against thanos components (querier here). What I am doing is,

1. Running adhoc k8s pod with thanosbench in kubernetes cluster under same monitor namespace where prom+sidecar, thanos components pods are running.
2. command I provided
   kubectl run -n monitor --rm -i --tty thanosbench --image=thanosbench:latest --restart=Never -- stress --workers=5 172.18.7.48:10901
   172.18.7.48 is the pod IP of thanos querier with 10901 container port for gRPC.

Error Log:
level=info ts=2020-09-14T20:52:09.182957888Z caller=stress.go:108 msg="stress test encountered an error" err="the StoreAPI responded with zero metric names" level=error ts=2020-09-14T20:52:09.183005192Z caller=main.go:114 msg="running command failed" err="the StoreAPI responded with zero metric names" pod "thanosbench" deleted pod monitor/thanosbench terminated (Error)

can anybody help me here what am I missing?
Also if you could explain where and how metrics names are looked by thanosbench, it would help my debugging.

[yeya24]

Hello @dmilind, I am wondering whether this relates to the latest change in Thanos or not. What Thanos version are you using?

[dmilind]

@yeya24 HI I am using 0.15.0 version of thanos

[yeya24]

I can reproduce this error now and I think this is caused by pr thanos-io/thanos#2964. I will open a fix pr soon.

[dmilind]

@yeya24 Awesome , thank you.

[dmilind]

Any update on this?

[yeya24]

Umm, I think it just needs another 👁️ to review that pr.