Skip to content

Project: Table entries in the viewer in SDKs #9455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
grtlr opened this issue Apr 2, 2025 · 1 comment
Closed

Project: Table entries in the viewer in SDKs #9455

grtlr opened this issue Apr 2, 2025 · 1 comment
Assignees
@grtlr
Labels
project Tracking issues for so-called "Projects"

Comments

@grtlr
Copy link
Contributor

grtlr commented Apr 2, 2025 •
edited
Loading

Summary description

Add support for Tables (read dataframes) as first-class entries in the Rerun viewer and Python SDK, enabling direct dataframe visualization and better integration with notebook workflows.

This issue is mostly AI generated 🤖. The original internal proposal was written by @abey79 and @grtlr.

Background and problem

The current chunk-centric storage/visualization infrastructure, while excellent for unaligned data, introduces unnecessary friction when dealing with simple dataframes:

  • Dataframes must be deconstructed to fit the current logging API
  • Complex blueprint creation is required to emulate dataframe display
  • The "recording" user experience isn't optimal for dataframe visualization

Grounding use-cases

  • Send a dataframe from a notebook/script to the viewer
  • Load and display a remote table in the viewer
  • Enhanced dataframe display compared to traditional notebook/HTML-based tools
  • Better integration with other viewer features

How to describe, demo and evaluate

Success metrics:

  • Implementation of a simplified but functional Table support in the viewer
  • Successful integration with Python SDK
  • Basic table visualization functionality
  • Support for atomic table operations (drop & replace)

Designs and plans

Key implementation components:

  • New RerunClient top-level object for viewer connection
    Table storage using Vec
  • Integration with existing StoreHub/StoreBundle architecture
  • UI updates including renamed "Stores" panel and table-specific viewing mode

Tasks

Non-goals and won't do

  • Support for tables that don't fit in memory
  • Complex table operations beyond drop & replace
  • Alternative table visualizations or blueprint support
  • Garbage collection for tables
  • Reading tables out of the viewer
  • Support for column hide/show/reordering, filtering, and searching in this phase
@grtlr grtlr added the project Tracking issues for so-called "Projects" label Apr 2, 2025
@grtlr grtlr self-assigned this Apr 2, 2025
This was referenced Apr 2, 2025
Closed
Merged
grtlr added a commit that referenced this issue Apr 9, 2025
@grtlr
Add experimental send_table to Python SDK (#9538)
efe29aa 
### Related

* Supersedes #9456
* Closes #9348
* Closes #8443
* Part of #8692
* Part of #9455

### What

This adds minimal support for `send_table` to `re_grpc_client` and
implements a new experimental API in the Python SDK:

```py
client = ViewerClient(addr="rerun+http://0.0.0.0:9876")
client.send_table(
    "Hello from Python",
    pa.RecordBatch.from_pydict({
        "id": [1, 2, 3],
        "url": ["https://www.rerun.io", "https://github.com/rerun-io/rerun", "https://crates.io/crates/rerun"],
    }),
)
```

### Testing

Either via the snippet:

```sh
pixi run py-build
pixi run -e py python docs/snippets/all/howto/send_table.py
```

Or via the notebook that was updated in this PR.
@grtlr
Copy link
Contributor Author

grtlr commented Apr 28, 2025

The last bits of this have been implemented as outlined in this issue via #9538.

@grtlr grtlr closed this as completed Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@grtlr grtlr

Labels
project Tracking issues for so-called "Projects"
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@grtlr