You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The chunks storage was transferring encoded chunks from ingesters to querier at query time, while the blocks storage is transferring decoded ones. Having to decode chunks in the ingesters increases the memory utilisation at query time and increases the likelihood of getting ingesters OOMKilled because of an heavy query.
Describe the solution you'd like
Historically, TSDB didn't allow to query for chunks, reason why we had to transfer decoded time-series from ingesters to querier, but now it's possible and we should it.
The text was updated successfully, but these errors were encountered:
Another side effect I've just learned about transferring decoded time-series from ingesters to querier is that protobuf unmarshalling is inefficient when transferring slices (repeated fields in protobuf). The problem is that, according to the doc, repeated fields don't carry the number of items in the serialized message and this prevents the unmarshalling from pre-allocating the slice with the right size when deserialising, causing a large number of growslice.
For example, this is the CPU profile in the querier while running an heavy query:
Is your feature request related to a problem? Please describe.
The chunks storage was transferring encoded chunks from ingesters to querier at query time, while the blocks storage is transferring decoded ones. Having to decode chunks in the ingesters increases the memory utilisation at query time and increases the likelihood of getting ingesters OOMKilled because of an heavy query.
Describe the solution you'd like
Historically, TSDB didn't allow to query for chunks, reason why we had to transfer decoded time-series from ingesters to querier, but now it's possible and we should it.
The text was updated successfully, but these errors were encountered: