Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade thanos that include skip bad blocks during compaction #4453

Closed
wants to merge 1 commit into from

Conversation

huyan0
Copy link

@huyan0 huyan0 commented Aug 31, 2021

Signed-off-by: Yang Hu [email protected]

What this PR does:
This PR is a follow up of #4423. Following the recommendation, I created this PR to update Thanos dependency before supporting skipping bad blocks with bucket index enabled and creating a feature flag.

Update Thanos version to include 4566 and 4469 to skip blocks with out-of-order chunks. The skip is currently disabled and will be enabled in a follow-up PR

Which issue(s) this PR fixes:
Step towards fixes #3569

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@bboreham
Copy link
Contributor

bboreham commented Sep 1, 2021

Includes Thanos changes since #4467

This isn't a good way to express the changes - "since" will change every day.
I can't see how the link under #4467 relates to what you did.
Can you list the key changes please?

Also put a short description in the CHANGELOG, mentioning the new metric.

@huyan0 huyan0 force-pushed the master branch 3 times, most recently from 6e3f5cf to f8119ca Compare September 1, 2021 18:53
@huyan0 huyan0 changed the title upgrade thanos that include skip bad blocks blocks during compaction upgrade thanos that include skip bad blocks during compaction Sep 2, 2021
CHANGELOG.md Outdated
@@ -1,7 +1,7 @@
# Changelog

## master / unreleased

* [CHANGE]: Compactor: Introduce a new metric to monitor blocks marked for no compact during compaction: cortex_compactor_blocks_marked_for_no_compact_total
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if we can briefly explain why when would cortex_compactor_blocks_marked_for_no_compact_total go up (i.e why would block get marked for no compact). This will give the Cortex admin, as well as customers reading the change log more context on the background of the metrics.


// TSDB syncer metrics
syncerMetrics *syncerMetrics
}

type noOpsCounter struct {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on sync offline, I have created this noOpsCounter so that there's no metric when feature is disabled. I don't think this is elegant and I'm looking at some examples. Would appreciate some advice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bboreham @pracucci @pstibrany it would be nice to have your thoughts regarding this as well - should we not emit the metric at all when "skip bad block" feature is disabled.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just pass prometheus.NewCounter(prometheus.CounterOpts{}) - if it isn't registered it doesn't go anywhere.

Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is failing; CHANGELOG still needed.


// TSDB syncer metrics
syncerMetrics *syncerMetrics
}

type noOpsCounter struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just pass prometheus.NewCounter(prometheus.CounterOpts{}) - if it isn't registered it doesn't go anywhere.

@hansh0801
Copy link

when this PR is going to be release?

@stale
Copy link

stale bot commented Jan 26, 2022

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 26, 2022
@nschad
Copy link
Contributor

nschad commented Feb 9, 2022

We also ran into the issue described by #3569. Is there something I can do to help with this PR?

@stale
Copy link

stale bot commented Jun 11, 2022

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 11, 2022
@shybbko
Copy link

shybbko commented Jun 11, 2022

@alanprot doesn't #4706 close this?

@stale stale bot removed the stale label Jun 11, 2022
@alanprot
Copy link
Member

I will close this as it was implemented on #4707

@alanprot alanprot closed this Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
7 participants