Skip to content

Stream chunks from ingesters to querier when running the blocks storage #2845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pracucci opened this issue Jul 6, 2020 · 7 comments
Closed
Labels
storage/blocks Blocks storage engine type/feature

Comments

@pracucci
Copy link
Contributor

pracucci commented Jul 6, 2020

When running a query in a Cortex cluster running the chunks storage, the querier fetches (encoded) chunks from ingesters and these chunks are decoded in the querier itself.

On the contrary, when running the blocks storage, the chunks are decoded in the ingester and the querier fetches (decoded) SeriesSet from ingesters. This puts extra pressure on the ingesters, based on the query QPS and the number of series touched by each query.

We want to stream chunks from ingesters to querier in the blocks storage too.

@pracucci pracucci added the storage/blocks Blocks storage engine label Jul 6, 2020
@stale
Copy link

stale bot commented Sep 4, 2020

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Sep 4, 2020
@pstibrany pstibrany removed the stale label Sep 4, 2020
@pracucci
Copy link
Contributor Author

pracucci commented Sep 14, 2020

Still valid. Now TSDB supports chunks-based querier/iterator, so we can build it.

@stale
Copy link

stale bot commented Nov 13, 2020

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 13, 2020
@pracucci
Copy link
Contributor Author

Still valid.

@stale stale bot removed the stale label Nov 14, 2020
@stale
Copy link

stale bot commented Mar 2, 2021

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 2, 2021
@jtlisi
Copy link
Contributor

jtlisi commented Mar 2, 2021

still valid

@jtlisi
Copy link
Contributor

jtlisi commented Mar 18, 2021

Closed by #3889

@jtlisi jtlisi closed this as completed Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
storage/blocks Blocks storage engine type/feature
Projects
None yet
Development

No branches or pull requests

3 participants