Skip to content

Tagged columnar updates: Python #8792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jan 27, 2025
Merged

Tagged columnar updates: Python #8792

merged 13 commits into from
Jan 27, 2025

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Jan 23, 2025

Implements everything needed for the new tagged send_columns in Python. This includes taking care of interactions with indicators, mono-components, etc.

def send_columns_v2(
    entity_path: str,
    indexes: Iterable[TimeColumnLike],
    columns: Iterable[ComponentColumn],
    recording: RecordingStream | None = None,
    strict: bool | None = None,
) -> None:

Points (multi-component):

# Prepare a point cloud that evolves over 5 timesteps, changing the number of points in the process.
times = np.arange(10, 15, 1.0)
# fmt: off
positions = [
    [1.0, 0.0, 1.0], [0.5, 0.5, 2.0],
    [1.5, -0.5, 1.5], [1.0, 1.0, 2.5], [-0.5, 1.5, 1.0], [-1.5, 0.0, 2.0],
    [2.0, 0.0, 2.0], [1.5, -1.5, 3.0], [0.0, -2.0, 2.5], [1.0, -1.0, 3.5],
    [-2.0, 0.0, 2.0], [-1.5, 1.5, 3.0], [-1.0, 1.0, 3.5],
    [1.0, -1.0, 1.0], [2.0, -2.0, 2.0], [3.0, -1.0, 3.0], [2.0, 0.0, 4.0],
]
# fmt: on

# At each timestep, all points in the cloud share the same but changing color and radius.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
radii = [0.05, 0.01, 0.2, 0.1, 0.3]

rr.send_columns_v2(
    "points",
    indexes=[rr.TimeSecondsColumn("time", times)],
    columns=[
        *rr.Points3D.columns(positions=positions).partition(lengths=[2, 4, 4, 3, 4]),
        *rr.Points3D.columns(colors=colors, radii=radii),
    ],
)

Scalar (mono-component):

times = np.arange(0, 64)
scalars = np.sin(times / 10.0)

rr.send_columns_v2(
    "scalars",
    indexes=[rr.TimeSequenceColumn("step", times)],
    columns=rr.Scalar.columns(scalar=scalars),
)

Custom data:

timestamps = np.arange(0, 64)

# Log two component columns, named "sin" and "cos", with the corresponding values
rr.send_columns_v2(
    "/",
    indexes=[rr.TimeSequenceColumn("step", timestamps)],
    columns=rr.AnyValues.columns(sin=np.sin(timestamps / 10.0), cos=np.cos(timestamps / 10.0)),
)

Copy link

github-actions bot commented Jan 23, 2025

Web viewer built successfully. If applicable, you should also test it:

  • I have tested the web viewer
Result Commit Link Manifest
c4f4441 https://rerun.io/viewer/pr/8792 +nightly +main

Note: This comment is updated whenever you push a commit.

@teh-cmc
Copy link
Member Author

teh-cmc commented Jan 23, 2025

@rerun-bot full-check

Copy link

Base automatically changed from cmc/fix_python_tagged_pipelines to main January 24, 2025 08:53
@teh-cmc teh-cmc force-pushed the cmc/tagged_columnar_apis_python branch from 17369d5 to d6dff74 Compare January 24, 2025 08:55
@teh-cmc teh-cmc marked this pull request as draft January 24, 2025 14:00
@teh-cmc
Copy link
Member Author

teh-cmc commented Jan 24, 2025

Drafting: upcoming agreed-upon design changes.

@teh-cmc teh-cmc force-pushed the cmc/tagged_columnar_apis_python branch from d6dff74 to 73cc40c Compare January 24, 2025 14:56
@teh-cmc teh-cmc force-pushed the cmc/tagged_columnar_apis_python branch 3 times, most recently from b371a63 to 1bbaba0 Compare January 24, 2025 16:55
@teh-cmc teh-cmc force-pushed the cmc/tagged_columnar_apis_python branch from 1bbaba0 to 5570f8b Compare January 24, 2025 16:58
@teh-cmc
Copy link
Member Author

teh-cmc commented Jan 24, 2025

@rerun-bot full-check

@teh-cmc teh-cmc marked this pull request as ready for review January 24, 2025 17:05
Copy link

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12954162407

@abey79 abey79 self-requested a review January 24, 2025 17:20
@teh-cmc teh-cmc removed the do-not-merge Do not merge this PR label Jan 27, 2025
Copy link
Member

@abey79 abey79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done, great API improvement!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why these are gone from the index?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it doesn't directly mention any of these components anymore (the parsing logic is very dumb).

It's fine though, it appears in the Features section already.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, now that I think about it... The entire Components section of the index should go away.

@teh-cmc teh-cmc merged commit 84405e7 into main Jan 27, 2025
32 checks passed
@teh-cmc teh-cmc deleted the cmc/tagged_columnar_apis_python branch January 27, 2025 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tagged columnar updates: Python
2 participants