Description
Background
The dataframe view has always been a bit of an outlier:
- It's (selection panel) UI doesn't fit well with the rest of the views.
- The timeline selection logic is different from the rest of the viewer.
- It can very quickly run into performance problem.
Also, it is more generally problematic because of the strong performance constraints it puts on the dataframe query engine, because the time budget in the context of the viewer is at least 2 orders of magnitude smaller than in other contexts (i.e. it's ok to wait 1s for a query in a notebook, but the dataframe view must render in less than 10ms).
Finally, the fact that the dataframe view is a odd fit introduces a lot of friction with maintenance, which impacts our ability to improve the dataframe query engine.
Proposal
Realising that the dataframe query engine is a better fit for one-shot queries (as opposed to continuously refreshing views), and noting that the viewer now supports tables, I propose to sunset the dataframe view and replace it with a "convert recording to table" feature:
- It would apply to whatever is the current recording.
- Upon activation, it would display a (modal?) UI exposing all the knobs of the dataframe query engine.
- Once validated, the query would be executed in a background tasks, which, upon completion, would store its result in a new local table entry.
- Stretch goal: the query could be "remembered", and associated with the resulting table, such that it could be manually refreshed by the user.
- Related: the
DatafustionTableWidget(which is currently used to display table entires) should have the ability to expand rows to display instances (like the current dataframe view is capable of).
An alternative to this proposal would be to maintain simplified version of the dataframe view, trying to balance its current usefulness (and usage, if it actually is substantial?) while reducing the maintenance frictions mentioned above.