feat(pipeline): Universal processor for LeRobot #1431
+4,612
−52
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a flexible, composable pipeline for processing robot environment transitions, replacing the previous hardcoded preprocessing and enabling key renaming, image/state transformation, normalization, and serialization via reusable steps.
- Adds
RobotProcessorandProcessorStepRegistryfor chaining and serializing arbitrary processing steps - Implements
RenameProcessor,ImageProcessor,StateProcessor,VanillaObservationProcessor, and normalization processors (ObservationNormalizer,ActionUnnormalizer,NormalizationProcessor) - Refactors
preprocess_observationto delegate toRobotProcessorand updates scripts/tests to use the new pipeline
Reviewed Changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/processor/test_rename_processor.py | New tests covering key-renaming with RenameProcessor |
| tests/processor/test_pipeline.py | Extensive tests for RobotProcessor, hooks, serialization, devices |
| tests/processor/test_observation_processor.py | Tests for image/state processing in VanillaObservationProcessor |
| tests/processor/test_normalize_processor.py | Tests for normalization and unnormalization processors |
| tests/policies/test_policies.py | Replaces direct preprocess_observation calls with pipeline usage |
| tests/envs/test_envs.py | Updates env tests to use RobotProcessor for preprocessing |
| src/lerobot/processor/pipeline.py | Implements core RobotProcessor and step registry |
| src/lerobot/processor/rename_processor.py | Adds RenameProcessor |
| src/lerobot/processor/observation_processor.py | Adds ImageProcessor, StateProcessor, VanillaObservationProcessor |
| src/lerobot/processor/normalize_processor.py | Adds normalization/un-normalization processors |
| src/lerobot/envs/utils.py | Refactors preprocess_observation to use the new pipeline |
| src/lerobot/scripts/eval.py | Updates rollout logic to use RobotProcessor for observation prep |
| src/lerobot/templates/robotprocessor_modelcard_template.md | Introduces model-card template |
| docs/source/processor_tutorial.mdx | Adds comprehensive pipeline tutorial |
| docs/source/_toctree.yml | Registers the new tutorial in the docs sidebar |
Comments suppressed due to low confidence (1)
src/lerobot/envs/utils.py:28
- [nitpick] The type annotation restricts inputs to
dict[str, np.ndarray], but downstream code accepts nested dicts and other types. Consider broadening the signature todict[str, Any]for accuracy.
def preprocess_observation(observations: dict[str, np.ndarray]) -> dict[str, Tensor]:
AdilZouitine
force-pushed
the
user/azouitine/2025-7-2-implement-pipeline
branch
from
c8522c4 to
fd6dd5a
Compare
- Introduced `RobotPipeline` and `ObservationProcessor` for handling observation transformations. - Updated `preprocess_observation` to maintain backward compatibility while leveraging the new pipeline. - Added tests for the new processing components and ensured they match the original functionality. - Removed hardcoded logic in favor of a more flexible, composable architecture.
for more information, see https://pre-commit.ci
- Updated `ObservationProcessor` to enhance the modular design for processing observations. - Cleaned up imports and improved code readability by removing unnecessary lines and comments. - Ensured backward compatibility while integrating new processing components. - Added tests to validate the functionality of the updated processing architecture.
… in `test_observation_processor.py` to streamline the test suite and improve maintainability.
- Replaced instances of RobotPipeline with RobotProcessor across the codebase for improved modularity and clarity. - Introduced ProcessorStepRegistry for better management of processing steps. - Updated relevant documentation and tests to reflect the new processing structure. - Enhanced the save/load functionality to support the new processor design. - Added a model card template for RobotProcessor to facilitate sharing and documentation.
- Introduced a new tutorial on using RobotProcessor for preprocessing robot data. - Added a section in the table of contents for easy navigation to the new tutorial. - The tutorial covers key concepts, real-world scenarios, and practical examples for effective use of the RobotProcessor pipeline.
for more information, see https://pre-commit.ci
- Introduced `NormalizationProcessor` to handle both observation normalization and action unnormalization. - Added `ObservationNormalizer` and `ActionUnnormalizer` classes for specific normalization tasks. - Updated `__init__.py` to include the new `NormalizationProcessor` in the module exports. - Enhanced `ObservationProcessor` with registration in the `ProcessorStepRegistry` for better modularity. - Created `RenameProcessor` for renaming keys in observations, improving flexibility in data processing.
for more information, see https://pre-commit.ci
- Added `RenameProcessor` to facilitate key renaming in observations, improving data handling flexibility. - Updated `__init__.py` to include `RenameProcessor` in module exports. - Refactored `NormalizationProcessor` and `ObservationNormalizer` to use `rsplit` for better key handling. - Introduced comprehensive tests for `NormalizationProcessor` and `RenameProcessor` to ensure functionality and robustness.
Co-authored-by: Copilot <[email protected]> Signed-off-by: Adil Zouitine <[email protected]>
AdilZouitine
force-pushed
the
user/azouitine/2025-7-2-implement-pipeline
branch
from
fd6dd5a to
e1a1340
Compare
- Renamed `ObservationNormalizer` to `NormalizerProcessor` and `ActionUnnormalizer` to `UnnormalizerProcessor` for clarity. - Consolidated normalization logic for both observations and actions into `NormalizerProcessor` and `UnnormalizerProcessor`. - Updated tests to reflect the new class names and ensure proper functionality of normalization and unnormalization processes. - Enhanced handling of missing statistics in normalization processes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RobotPipelineandObservationProcessorfor handling observation transformations.preprocess_observationto maintain backward compatibility while leveraging the new pipeline.What this does
Explain what this PR does. Feel free to tag your PR with the appropriate label(s).
Examples:
How it was tested
Explain/show how you tested your changes.
Examples:
test_somethingintests/test_stuff.py.new_featureand checked that training converges with policy X on dataset/environment Y.some_function, it now runs X times faster than previously.How to checkout & try? (for the reviewer)
Provide a simple way for the reviewer to try out your changes.
Examples:
SECTION TO REMOVE BEFORE SUBMITTING YOUR PR
Note: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.
Note: Before submitting this PR, please read the contributor guideline.