Open
Description
Feature request
LSDB.nested NestedFrame parquet I/O is currently not updated to be able to serialize nested columns on read or write, like nested-pandas and LSDB.catalog I/O both now can.
I think this is low priority, but we should update these at some point. See this closed PR for an 80% implementation: #727
This will just limit the ability for users to write more general parquet through the _ddf interface, for example:
# writing out
cat._ddf.to_parquet("catalog_as_normal_df.parquet")
# reading in
from lsdb.nested import read_parquet
ndf = read_parquet("catalog_as_normal_df.parquet")
My assumption is that the above is not really that important to have right now, and I could even see an argument being made to limit this ability all together.
Before submitting
Please check the following:
- I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
- I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
- If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
No status