query-frontend: add option to preserve specific request headers

[srijan55]

What this PR does:
When applying PrometheusCodec headers from the requests are not transferred, which makes using Cortex in frontend-only mode with any other Prometheus engine harder. This PR fixes this issue.

Which issue(s) this PR fixes:
Fixes #4416

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[srijan55]

@pstibrany maybe you can have a look or assign it to someone who is the best for the area? I reached out on the slack channel as well, but couldn't get engagement on this.

[bboreham]

Thanks; some thoughts below.

[srijan55]

@bboreham Thanks for your comments. Incorporated changes. Please review.

[srijan55]

@bboreham Thanks for your comments. Incorporated changes. Please review.

@bboreham , friendly ping again on this. Hope you can find a moment to review and merge

[srijan55]

@bboreham , Thanks for another round of comments. Addressed all of them. Please review and help with the merge when you have a moment.

[srijan55]

@bboreham , I wasn't able to reproduce the integration tests failing, so I have pushed another commit in hope that it might help. Can you kick off the workflows again.

[srijan55]

@bboreham, please help review /merge again. I figure out how to run integration tests and fixed the issue. They should all be green now. Thank you .

[srijan55]

Thanks for valuable comments.

[bboreham]

Hi, we discussed this at the Cortex community call today; the general feeling was that rather than passing all headers and having a specific block-list, Cortex should drop all headers by default and let the administrator configure an allow-list.

Could you have a go at that please?

[srijan55]

Hi, we discussed this at the Cortex community call today; the general feeling was that rather than passing all headers and having a specific block-list, Cortex should drop all headers by default and let the administrator configure an allow-list.

Could you have a go at that please?

Sure. I'll work on that. Would you prefer the new config setting to be in frontend subsection or querier or somewhere else?

[srijan55]

Hi, we discussed this at the Cortex community call today; the general feeling was that rather than passing all headers and having a specific block-list, Cortex should drop all headers by default and let the administrator configure an allow-list.

Could you have a go at that please?

Did the change. Can you help quickly review, so I know I haven't missed obvious things. This is first contribution so still learning.

[bboreham]

Getting there. I have a number of suggestions to word things differently, and one more copy of an object that I don't understand.

[srijan55]

Getting there. I have a number of suggestions to word things differently, and one more copy of an object that I don't understand.

Please have a look again, when you get a moment.

[pracucci]

Makes sense to me. I left a non blocking comment. Thanks!

[bboreham]

LGTM! Thanks for this Manish.