mmcinnestaylor / thriftysirius

CS 391R: Robot Learning @ UT Austin course project

Sirius 🌟: Robot Learning on the Job

This is the official codebase for the Sirius paper:

Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment
Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, Yuke Zhu
UT Austin Robot Perception and Learning Lab
Robotics: Science and Systems (RSS), 2023
[Paper]  [Project Website]  [Real Robot Control]

Quickstart

Sirius builds upon robomimic, a framework for robot learning from demonstration. Sirius also uses the robotics simulator robosuite powered by the MuJoCo physics engine.

Setup Sirius codebase

Installing Sirius

git clone https://github.com/UT-Austin-RPL/sirius
cd sirius
conda env create -f sirius.yml
conda activate sirius
pip install -e .

Installing robosuite

The additional reference for installing robosuite here and here could be helpful.

$ git clone https://github.com/ARISE-Initiative/robosuite.git
$ cd robosuite
$ pip install -r requirements.txt
$ pip install -e .

Usage

Running Sirius

Running Sirius intervention-guided policy learning:

python robomimic/scripts/train.py --config robomimic/exps/sirius/sirius.json

IWR baseline:

python robomimic/scripts/train.py --config robomimic/exps/sirius/bc_iwr.json

BC baseline:

python robomimic/scripts/train.py --config robomimic/exps/sirius/bc.json

Sirius Data Collection Pipeline

We include the script for collecting demonstrations and performing human intervention during robot policy execution below. We use a spacemouse for providing both demonstration and intervention. More details for setting up Spacemouse can be found here.

Performing Human Demonstration

Perform human demonstration with the flag --all-demos:

python robomimic/scripts/hitl/collect_hitl_demos.py --all-demos --num-traj 50

Policy Execution with Intervention

Perform human intervention with the policy checkpoint ${checkpoint}:

python robomimic/scripts/hitl/collect_hitl_demos.py --num-traj 50 --checkpoint ${checkpoint}

Processing data

Adding modalities

By default, the datasets are generated in the minimum format with only low-level state information to save space. To add image observation and other modalities for training, run the following post-processing script. It will process the original data ${data.hdf5} into ${data_processed.hdf5}, with image size ${image_size}. By default, the two camera view uses are agentview and robot0_eye_in_hand, which you can modify in the script template_process_sim_dataset.sh.

cd robomimic/scripts/hitl

source template_process_sim_dataset.sh ${data.hdf5} ${data_processed.hdf5} ${image_size}

Acknowledgements

This codebase is largely built on robomimic and robosuite. We also thank Ajay Mandlekar for sharing well-designed simulation task environments beyond the robomimic codebase like Coffee and Threading tasks during project development.

For real-robot experiments, we used Deoxys, a controller library for Franka Emika Panda developed by Yifeng Zhu.

Citation

@inproceedings{liu2022robot,
    title = {Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment},
    author = {Huihan Liu and Soroush Nasiriany and Lance Zhang and Zhiyao Bao and Yuke Zhu},
    booktitle = {Robotics: Science and Systems (RSS)},
    year = {2023}
}