Skip to content

feat(meta): add system parameter pause_on_next_bootstrap #11936

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Sep 1, 2023

Conversation

BugenZhao
Copy link
Member

@BugenZhao BugenZhao commented Aug 29, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Add system parameter pause_on_next_bootstrap. When set to true, the cluster will pause all data sources (including connectors and DMLs) on the next startup, then reset this value. This is for debugging and maintenance purposes.

To resume the data, users have to run risectl meta resume, or simply restart the cluster again. We may introduce an SQL command to do this in the future.

By introducing the PausedReason in the global barrier manager, the paused state won't be accidentally broken by the Resume stage of scaling, or creating a new streaming job.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

  • Add a new system parameter pause_on_next_bootstrap. When set to true, the cluster will pause all data sources (including connectors and DMLs) on the next startup, then reset this value. This is for debugging and maintenance purposes.

Signed-off-by: Bugen Zhao <[email protected]>
Signed-off-by: Bugen Zhao <[email protected]>
Signed-off-by: Bugen Zhao <[email protected]>
Signed-off-by: Bugen Zhao <[email protected]>
Signed-off-by: Bugen Zhao <[email protected]>
@github-actions github-actions bot added the type/feature Type: New feature. label Aug 29, 2023
@BugenZhao
Copy link
Member Author

Will write a deterministic test to cover this.

Signed-off-by: Bugen Zhao <[email protected]>
Copy link
Collaborator

@fuyufjh fuyufjh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 4023 files.

Valid Invalid Ignored Fixed
1774 1 2248 0
Click to see the invalid file list
  • src/tests/simulation/tests/integration_tests/recovery/pause_on_bootstrap.rs

Signed-off-by: Bugen Zhao <[email protected]>
Signed-off-by: Bugen Zhao <[email protected]>
@BugenZhao BugenZhao enabled auto-merge September 1, 2023 10:22
@BugenZhao BugenZhao added this pull request to the merge queue Sep 1, 2023
@codecov
Copy link

codecov bot commented Sep 1, 2023

Codecov Report

Merging #11936 (5dd79c1) into main (1217290) will decrease coverage by 0.02%.
The diff coverage is 47.58%.

@@            Coverage Diff             @@
##             main   #11936      +/-   ##
==========================================
- Coverage   69.81%   69.80%   -0.02%     
==========================================
  Files        1392     1392              
  Lines      233462   233510      +48     
==========================================
+ Hits       162996   163006      +10     
- Misses      70466    70504      +38     
Flag Coverage Δ
rust 69.80% <47.58%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
src/meta/src/barrier/schedule.rs 80.50% <0.00%> (-1.39%) ⬇️
src/meta/src/rpc/service/scale_service.rs 0.00% <0.00%> (ø)
src/meta/src/stream/scale.rs 10.51% <0.00%> (ø)
src/meta/src/stream/stream_manager.rs 74.01% <0.00%> (ø)
src/stream/src/executor/dml.rs 89.47% <ø> (ø)
...c/stream/src/executor/source/fs_source_executor.rs 0.00% <ø> (ø)
src/meta/src/barrier/command.rs 34.47% <28.94%> (-0.85%) ⬇️
src/stream/src/executor/mod.rs 47.57% <33.33%> (-0.45%) ⬇️
src/meta/src/model/barrier.rs 76.00% <60.00%> (-24.00%) ⬇️
src/meta/src/barrier/recovery.rs 52.56% <61.11%> (-0.10%) ⬇️
... and 8 more

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Merged via the queue into main with commit 87af7a2 Sep 1, 2023
@BugenZhao BugenZhao deleted the bz/pause-on-bootstrap branch September 1, 2023 10:55
BugenZhao added a commit that referenced this pull request Sep 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature Type: New feature.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants