Skip to content

refactor(stream): do not broadcast actor info in shared context #21258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 10, 2025

Conversation

wenym1
Copy link
Contributor

@wenym1 wenym1 commented Apr 7, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

Highest Release Version: 2.3

What's changed and what's your intention?

Previously we broadcast the ActorInfo of globally all running actors to the SharedContext of each CN, so that the DispatchExecutor and MergeExecutor can get the host of downstream or upstream actors to establish the exchange connection.

In #21256, we already unified remote and local output so that the DispatchExecutor won't need the ActorInfo of downstream actor. In this PR, instead of only providing with upstream actor_ids, we will change to provide the upstream ActorInfo directly for MergeExecutor, either in build actor info, or MergeUpdate, so that MergeExecutor also won't need to get ActorInfo from SharedContext, and in the end, we don't have to broadcast and maintain the global list of ActorInfo anymore.

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

@wenym1 wenym1 force-pushed the yiming/shared-context-in-local-barrier-manager branch from ed0787f to 0baa530 Compare April 7, 2025 09:52
@wenym1 wenym1 force-pushed the yiming/no-broadcast-actor-info branch from fa98c2a to 0c7cb53 Compare April 7, 2025 09:53
@wenym1 wenym1 force-pushed the yiming/shared-context-in-local-barrier-manager branch from 0baa530 to c14464f Compare April 8, 2025 07:44
@wenym1 wenym1 force-pushed the yiming/no-broadcast-actor-info branch 2 times, most recently from 3fcc0c8 to 9a664fd Compare April 8, 2025 08:42
Base automatically changed from yiming/shared-context-in-local-barrier-manager to main April 8, 2025 09:47
@wenym1 wenym1 force-pushed the yiming/no-broadcast-actor-info branch from 9a664fd to 4806391 Compare April 8, 2025 11:38
Copy link
Member

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wenym1 wenym1 force-pushed the yiming/no-broadcast-actor-info branch from 4806391 to 429a69c Compare April 9, 2025 11:36
@wenym1 wenym1 enabled auto-merge April 9, 2025 11:53
@wenym1 wenym1 disabled auto-merge April 9, 2025 11:54
@wenym1 wenym1 added this pull request to the merge queue Apr 10, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 10, 2025
@wenym1 wenym1 added this pull request to the merge queue Apr 10, 2025
Merged via the queue into main with commit 52eb7c2 Apr 10, 2025
32 of 34 checks passed
@wenym1 wenym1 deleted the yiming/no-broadcast-actor-info branch April 10, 2025 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants