Skip to content

Add chunk skipping GUC #7360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 18, 2024
Merged

Conversation

antekresic
Copy link
Member

Add the ability to enable/disable new chunk skipping functionality completely.

@antekresic antekresic added this to the TimescaleDB 2.17.1 milestone Oct 18, 2024
@antekresic antekresic self-assigned this Oct 18, 2024
@antekresic antekresic force-pushed the add_chunk_skipping_guc branch 2 times, most recently from 4f7b940 to 95e9f9b Compare October 18, 2024 09:16
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

Attention: Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 82.57%. Comparing base (59f50f2) to head (d791330).
Report is 1009 commits behind head on main.

Files with missing lines Patch % Lines
src/ts_catalog/chunk_column_stats.c 25.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7360      +/-   ##
==========================================
+ Coverage   80.06%   82.57%   +2.51%     
==========================================
  Files         190      229      +39     
  Lines       37181    42678    +5497     
  Branches     9450    10723    +1273     
==========================================
+ Hits        29770    35243    +5473     
- Misses       2997     3160     +163     
+ Partials     4414     4275     -139     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add the ability to enable/disable new chunk skipping functionality
completely.
@antekresic antekresic force-pushed the add_chunk_skipping_guc branch from 95e9f9b to d791330 Compare October 18, 2024 13:36
@antekresic antekresic added the force-auto-backport Automatically backport this PR or fix of this issue, even if it's not marked as "bug" label Oct 18, 2024
@antekresic antekresic enabled auto-merge (rebase) October 18, 2024 13:46
@antekresic antekresic merged commit 8767565 into timescale:main Oct 18, 2024
48 checks passed
@timescale-automation
Copy link
Member

Automated backport to 2.17.x not done: cherry-pick failed.

Git status

HEAD detached at origin/2.17.x
You are currently cherry-picking commit 8767565e3.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   .unreleased/pr_7360
	modified:   src/guc.h
	modified:   src/hypertable.c
	modified:   src/hypertable_restrict_info.c
	modified:   src/nodes/chunk_append/exec.c
	modified:   src/ts_catalog/chunk_column_stats.c
	modified:   tsl/test/expected/chunk_column_stats-14.out
	modified:   tsl/test/expected/chunk_column_stats-15.out
	modified:   tsl/test/expected/chunk_column_stats-16.out
	modified:   tsl/test/expected/chunk_column_stats-17.out
	modified:   tsl/test/sql/chunk_column_stats.sql.in

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   src/guc.c


Job log

@timescale-automation timescale-automation added auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts) backported-2.17.x labels Oct 18, 2024
fabriziomello added a commit to fabriziomello/timescaledb that referenced this pull request Oct 21, 2024
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#7360 Add chunk skipping GUC

**Bugfixes**
* timescale#7335 Change log level used in compression
* timescale#7342 Fix collation for in-memory tuple filtering

**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
@fabriziomello fabriziomello mentioned this pull request Oct 21, 2024
fabriziomello added a commit that referenced this pull request Oct 21, 2024
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* #7360 Add chunk skipping GUC

**Bugfixes**
* #7335 Change log level used in compression
* #7342 Fix collation for in-memory tuple filtering

**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
fabriziomello added a commit to fabriziomello/timescaledb that referenced this pull request Oct 21, 2024
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#7360 Add chunk skipping GUC

**Bugfixes**
* timescale#7335 Change log level used in compression
* timescale#7342 Fix collation for in-memory tuple filtering

**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
fabriziomello added a commit to fabriziomello/timescaledb that referenced this pull request Oct 21, 2024
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#7360 Add chunk skipping GUC

**Bugfixes**
* timescale#7335 Change log level used in compression
* timescale#7342 Fix collation for in-memory tuple filtering

**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
fabriziomello added a commit that referenced this pull request Oct 21, 2024
This release contains performance improvements and bug fixes since
the 2.17.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* #7360 Add chunk skipping GUC

**Bugfixes**
* #7335 Change log level used in compression
* #7342 Fix collation for in-memory tuple filtering

**Thanks**
* @gmilamjr for reporting an issue with the log level of compression messages
* @hackbnw for reporting an issue with collation during tuple filtering
@jflambert
Copy link

Sorry @fabriziomello and @svenklemm but I have a few questions

Why is it desirable to disable skipping chunks in v2.17.1? Are there downsides I should be aware of?

@fabriziomello
Copy link
Member

Sorry @fabriziomello and @svenklemm but I have a few questions

Why is it desirable to disable skipping chunks in v2.17.1? Are there downsides I should be aware of?

We noticed some CPU spikes in our fleet due to extra catalog lookups at planning time introduced by this feature. Usually such kind of feature is added behing a feature flag to control the usage but we missed it when the feature was developed. As the actual fix isn't something trivial and obvious we decided to add the feature flag for now and then have time to rework it and fix it.

@akuzm akuzm added released-2.18.0 Released in 2.18.0 released-2.17.1 Released in 2.17.1 labels May 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport-not-done Automated backport of this PR has failed non-retriably (e.g. conflicts) backported-2.17.x force-auto-backport Automatically backport this PR or fix of this issue, even if it's not marked as "bug" hypertable planner released-2.17.1 Released in 2.17.1 released-2.18.0 Released in 2.18.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants