Skip to content

opt(frontend): Permute state keys that contain Watermark column #6452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #6042
jon-chuang opened this issue Nov 18, 2022 · 0 comments
Closed
Tracked by #6042

opt(frontend): Permute state keys that contain Watermark column #6452

jon-chuang opened this issue Nov 18, 2022 · 0 comments
Assignees
Labels
type/feature Type: New feature.
Milestone

Comments

@jon-chuang
Copy link
Contributor

jon-chuang commented Nov 18, 2022

If there is no requirement on the ordering of a state key, we may want to permute keys for stateful operators so that watermark columns are the first column. This allows us to easily perform state cleaning in downstream operators such as HashAgg and HashJoin.

To achieve this, we may have an optimization pass that walks the child nodes of a Watermark-emitting operator such as WatermarkFilter and attempt to permute the group by and join keys.

@jon-chuang jon-chuang added the type/feature Type: New feature. label Nov 18, 2022
@github-actions github-actions bot added this to the release-0.1.15 milestone Nov 18, 2022
@jon-chuang jon-chuang changed the title opt(frontend): Permute primary keys that contain Watermark column opt(frontend): Permute state keys that contain Watermark column Nov 18, 2022
@jon-chuang jon-chuang mentioned this issue Nov 18, 2022
22 tasks
@st1page st1page modified the milestones: release-0.1.16, release-0.1.17 Feb 1, 2023
@st1page st1page modified the milestones: release-0.18, release-0.19 Mar 20, 2023
@st1page st1page closed this as completed Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature Type: New feature.
Projects
None yet
Development

No branches or pull requests

3 participants