Add frame level task

[Cadene]

What this does

Motivations: We want to start supporting datasets with multiple tasks per episode. We also want to iterate on a simpler API for adding new frames to a dataset, where all features are added using dataset.add_frame(frame: dict).

For instance:

• 2D features like a sequence of x,y positions (waypoints)
• string features like captions

TODO

• Add annotation per frame
• Remove task argument of save_episode . add_frame is now the only way to add the task (even if it's the same for every frame)
• Add first test for add_frame
• Updated examples/port_datasets/pusht_zarr.py accordingly

How it was tested

• Ran python examples/port_datasets/pusht_zarr.py
• Ran tests

How to checkout & try? (for the reviewer)

pytest -sx tests/test_datasets.py::test_add_frame

[aliberts]

LGTM with some comments

[aliberts]

Equivalent but a bit faster and easier to read I think

Suggested change

	task_index = self.get_task_index(task)
	if task_index not in self.tasks:
	if task not in self.tasks.values():
	task_index = self.get_task_index(task)

[Cadene]

I found this logic quite confusing, so I refactor it.

[aliberts]

This test is quite long for what it's doing (~0.5s)
Did you check if this new add_frame is slower than before?

[Cadene]

0.13s on my side

[Cadene]

reverting commit:

commit fd6436af9de41a8436396afc296ebb53b0d7bc2c (HEAD -> user/rcadene/2025_01_27_dataset_v2.1)
Author: Remi Cadene <[email protected]>
Date:   Thu Feb 13 19:05:20 2025 +0100

    Revert "Update tests/test_datasets.py"

    This reverts commit 16bb53f8d6e89214b83d3f706e86f40f9128c5f7.

commit 3abc897ca23329301333aec3ee085ca732e99679
Author: Remi Cadene <[email protected]>
Date:   Thu Feb 13 19:05:14 2025 +0100

    Revert "Update tests/test_datasets.py"

    This reverts commit 7170819c605a429402aed904d868b5eff21c0d2a.

commit 7170819c605a429402aed904d868b5eff21c0d2a (origin/user/rcadene/2025_01_27_dataset_v2.1)
Author: Remi <[email protected]>
Date:   Thu Feb 13 18:55:43 2025 +0100

    Update tests/test_datasets.py

    Co-authored-by: Simon Alibert <[email protected]>

commit 16bb53f8d6e89214b83d3f706e86f40f9128c5f7
Author: Remi <[email protected]>
Date:   Thu Feb 13 18:31:51 2025 +0100

    Update tests/test_datasets.py

    Co-authored-by: Simon Alibert <[email protected]>

Because getting:

__________________________________________________________________________________________________ test_add_frame_no_task ___________________________________________________________________________________________________

tmp_path = PosixPath('/private/var/folders/kj/4wjq7mtn6xdb3hz4bwd4db_c0000gn/T/pytest-of-rcadene/pytest-81/test_add_frame_no_task0')

    def test_add_frame_no_task(tmp_path):
        features = {"1d": {"dtype": "float32", "shape": (1,), "names": None}}
>       dataset = LeRobotDataset.create(repo_id=DUMMY_REPO_ID, fps=30, root=tmp_path, features=features)

tests/test_datasets.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lerobot/common/datasets/lerobot_dataset.py:957: in create
    obj.meta = LeRobotDatasetMetadata.create(
lerobot/common/datasets/lerobot_dataset.py:296: in create
    obj.root.mkdir(parents=True, exist_ok=False)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = PosixPath('/private/var/folders/kj/4wjq7mtn6xdb3hz4bwd4db_c0000gn/T/pytest-of-rcadene/pytest-81/test_add_frame_no_task0'), mode = 511, parents = True, exist_ok = False

    def mkdir(self, mode=0o777, parents=False, exist_ok=False):
        """
        Create a new directory at this given path.
        """
        try:
>           self._accessor.mkdir(self, mode)
E           FileExistsError: [Errno 17] File exists: '/private/var/folders/kj/4wjq7mtn6xdb3hz4bwd4db_c0000gn/T/pytest-of-rcadene/pytest-81/test_add_frame_no_task0'

../../miniconda3/envs/lerobot/lib/python3.10/pathlib.py:1175: FileExistsError