FACTR Teleop: Low-Cost Force-Feedback Teleoperation

Jason Jingzhou Liu^*, Yulong Li^*, Kenneth Shaw, Tony Tao, Ruslan Salakhutdinov, Deepak Pathak

Carnegie Mellon University

Project Page | arXiV | FACTR | FACTR Hardware

Catalog

• Installation
• FACTR Teleop
• Data Collection
• Training and Deployment
• License and Acknowledgements
• Citation

Installation

This repository requires ROS 2. If you have not installed ROS 2 yet, follow the official ROS 2 installation guide.

Provided ROS 2 Packages

The following ROS 2 packages are included in this repository:

• factr_teleop
• bc
• cameras
• python_utils

These packages are located in:

<repo_root>/src

ROS 2 Workspace Setup

These packages must reside within a ROS 2 workspace. If you do not already have one, create a workspace by following the ROS 2 workspace tutorial.

Then:

1. Copy the four provided packages into your workspace's src/ directory.
2. Ensure to source the ROS2 setup script in your terminal

   source /opt/ros/<ROS-Distribution>/setup.bash

   Note that this command should be run everytime you open a new terminal.
3. From the root of your workspace, build the workspace via:

   colcon build --symlink-install

   This should create the following folders in your workspace root

   build  install  log  src

4. From the root of your workspace, source the overlay via

   source install/local_setup.bash

   Note that this command should also be run everytime you open a new terminal.

For more guidance, refer to the ROS 2 Tutorial.

Additional Python Dependencies

Install ZMQ:

pip install zmq

Install Pinocchio:

sudo apt install ros-<ROS-Distribution>-pinocchio

• For example,

  sudo apt install ros-humble-pinocchio

Alternatively, try the following via pip.

python -m pip install pin

Finally, navigate to the Dynamixel submodule and install it via:

cd <repo_root>/src/factr_teleop/factr_teleop/dynamixel
pip install -e python

FACTR Teleop

Instructions for setting up FACTR leader arms and running the provided example demos can be found here.

Data Collection

We provide instructions and sample data collection scripts in ROS2. You might need your custom nodes for robots and sensors to run the system. In our case, the collected data is saved in following format:

Data Structure

Each trajectory is saved as a separate pickle file. Each pickle file contains a dictionary with the following structure:

trajectory.pkl
├── "data" : dict
│   ├── "topic_name_1" : list[data_points]
│   ├── "topic_name_2" : list[data_points]
│   └── ...
└── "timestamps" : dict
    ├── "topic_name_1" : list[timestamps]
    ├── "topic_name_2" : list[timestamps]
    └── ...

Key Components:

• data: A dictionary where:

  • Keys are the data source names (ROS topic names in our implementation)
  • Values are lists containing the actual data points (low-dimensional states or images)

• timestamps: A dictionary where:

  • Keys are the same data source names as in the "data" dictionary
  • Values are lists containing the timestamps when each corresponding data point was recorded

Note: Different data sources may log at different frequencies, resulting in varying list lengths across data sources. The timestamps are crucial for properly aligning and post-processing the data. While ROS provides synchronization APIs, we chose to record raw timestamps and perform post-processing to allow for greater flexibility in data analysis and alignment.

# Example of a trajectory structure
{
    "data": {
        "/camera/rgb/image_raw": [image1, image2, ...],
        "/joint_states": [state1, state2, ...],
        "/robot/end_effector_pose": [pose1, pose2, ...]
    },
    "timestamps": {
        "/camera/rgb/image_raw": [1615420323.45, 1615420323.55, ...],
        "/joint_states": [1615420323.40, 1615420323.50, ...],
        "/robot/end_effector_pose": [1615420323.42, 1615420323.52, ...]
    }
}

Training and Deployment

Data Processing and Training

Please check for detailed instructions in our factr repo.

Policy Rollout

We provide a sample rollout script in ROS2. In our case, the rollout launch file could be called as follows:

ros2 launch factr_teleop/launch/rollout.py

Please checkout rollout.py for details about configurations.

License and Acknowledgements

This source code is licensed under the Apache 2.0 liscence found in the LICENSE file in the root directory of this repository.

This project builds on top of or utilizes the following third party dependencies.

• GELLO: Inpiration for this work.
• ZMQ: Light-weight communication between python processes.
• Pinocchio: Fast kinematics and dynamics computation for manipulation.

Citation

If you find this codebase useful, feel free to cite our work!

@article{factr,
  title={FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning},
  author={Liu, Jason Jingzhou and Li, Yulong and Shaw, Kenneth and Tao, Tony and Salakhutdinov, Ruslan and Pathak, Deepak},
  journal={arXiv preprint arXiv:2502.17432},
  year={2025}
}