-
Notifications
You must be signed in to change notification settings - Fork 927
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add warning for poor compression ratio #7756
base: main
Are you sure you want to change the base?
Conversation
This change breaks roughly half our tests, since compressing chunks with only a few rows leads to a bad compression ratio. I considered adding this behind a GUC, which should probably be on by default, but except for testing I don't think there's ever a case where we want it to be OFF. We can either:
However, if we'd like to do more than just emit a warning in the future, then perhaps a GUC makes more sense, since that is something users might like to turn off. What are your thoughts on this @antekresic? |
076e1b0
to
cabda33
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7756 +/- ##
==========================================
+ Coverage 80.06% 81.92% +1.85%
==========================================
Files 190 247 +57
Lines 37181 45390 +8209
Branches 9450 11343 +1893
==========================================
+ Hits 29770 37187 +7417
- Misses 2997 3733 +736
- Partials 4414 4470 +56 ☔ View full report in Codecov by Sentry. |
cabda33
to
1889c9b
Compare
Adds functionality to emit a warning to the user when the compression ratio is "bad". Currently, "bad" is defined to be when size after compression is higher than after compression.
1889c9b
to
b8f1a65
Compare
compression_ratio < POOR_COMPRESSION_THRESHOLD ? | ||
WARNING : | ||
DEBUG1, | ||
"size before compression: " INT64_FORMAT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This probably needs better wording, its too dry and doesn't even include name of the chunk.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree with ante here that this needs a better error message
poor compression rate detected in chunk "xyz"
, the numbers could be moved into errdetail. Additionally you should probably use ereport here
Adds functionality to emit a warning to the user when the compression
ratio is "bad".
Currently, "bad" is defined to be when size after compression is higher
than after compression.