Gym environments for reinforcement learning based on the SOFA Simulation Framework. This repository is part of "LapGym - An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery". See LapGym for the paper and lap_gym for the top level repository.
Tested under Ubuntu {18.04, 20.04, 22.04}, Fedora {36, 37}, and Windows (WSL). MacOS (on Intel and Apple Silicon) is technically supported, however it currently does not work due to a cmake problem in the pybind11 part of SofaPython3.
- Follow the instructions to install SOFA and SofaPython3
- Install the
sofa_env
package
conda activate sofa
pip install -e .
- Test the installation
python3 sofa_env/scenes/controllable_object_example/controllable_env.py
This repository currently contains 14 reinforcement learning environments.
- 12 robot-assited laparoscopy environments from LapGym
- a reinforcement learning environment around the scene from A Simulation Framework for Magnetic Continuum Robots
- the reinforcement learning environment from Sim-to-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery
A SofaEnv
base class and sofa_templates
are provided to simplify implementing new reinforcement environments.
Please refer to the included documentation for a tutorial on how to build new scenes in SOFA and implement reinforcement environments with sofa_env
.
Also visit sofa_godot for SofaGodot, a Godot plugin to visually create new SOFA scenes.
You can either access the documentation here or build it locally with
cd docs
make html
firefox build/html/index.html
If you use the project in your work, please consider citing it with:
@article{JMLR:v24:23-0207,
author = {Paul Maria Scheikl and Balázs Gyenes and Rayan Younis and Christoph Haas and Gerhard Neumann and Martin Wagner and Franziska Mathis-Ullrich},
title = {LapGym - An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery},
journal = {Journal of Machine Learning Research},
year = {2023},
volume = {24},
number = {368},
pages = {1--42},
url = {http://jmlr.org/papers/v24/23-0207.html}
}
The Freemotion Animation Loop of SOFA solves collisions and other constraints with Lagrange Multipliers.
This method requires the computation of a compliance matrix for each object.
This compliance matrix can be precomputed and is valid as long as the object does not change too much for example through cutting.
Precomputing, handling, and loading the compliance matrix is done by the PrecomputedConstraintCorrection
object.
If your scene has one or more of them, the object will try to load the matrix from storage, and compute it, when it cannot find the appropriate file.
This can be seen by the terminal output of SOFA when creating the scene.
You may see a [ERROR] [FileRepository] File tissue-1479-0.1.comp NOT FOUND in: ...
output which is actually not an error but SOFA telling you it is now generating the precomputed compliance matrix.
This may take up to a few minutes.
- To get a general understanding of how SOFA works, read all pages under Simulation Principles in the official documentation
- To get a deeper understanding of the underlying principles, like mappings, read the SOFA Paper. But be aware, that some terms like MechanicalState (now called MechanicalObject) in the paper have changed in the actual API. Use it more as a "how does SOFA work?" not as a "how is SOFA implemented?"
- Look through the SOFA Tutorials. Some of the Tutorials might not work. That is generally not your fault. You can run them via the command line. For example
$FOLDER_TARGET/install/bin/runSofa $FOLDER_SRC/examples/Tutorials/Basic/TutorialBasicCube.scn
This work is supported by the Helmholtz Association under the joint research school "HIDSS4Health – Helmholtz Information and Data Science School for Health".