louvreaux / sphero_formation

RL Sphero control

NOTICE: This is a work in progress.

This package directly adds onto: https://github.com/mkrizmancic/sphero_formation/tree/0ac14aad3dd1a0af26f191c017e213279eebd52e It additonaly uses:

1. open_ai_ros: http://wiki.ros.org/openai_ros
2. DDPG-tf2: https://github.com/samuelmat19/DDPG-tf2
3. mantis_ddqn_navigation: https://github.com/bhctsntrk/mantis_ddqn_navigation

Dependencies

python 3.6
gym
tensorflow 2.x
xdotool
wmctrol

About

This package connects Stage simulator with RL algorithms. It contains the following scripts:

1. stage_connection.py - class that contains pause, unpause and reset functions for Stage
2. robot_stage_env.py - the most basic implementation of the RL environment
3. sphero_env.py - ROS-Sphero connection, inherits class from robot_stage_env.py
4. sphero_world.py - defines the task of the Sphero robot, inherits class from sphero_env.py
5. sphero_qlearn.py, sphero_dqn.py, sphero_ddpg.py - RL algorithms

Run

First of all, go to the package directory:

cd catkin_ws/src/sphero_formation

and initialize the updated stage_ros:

git submodule update --init

Copy updated Stage to /src:

cp -r ~/catkin_ws/src/sphero_formation/stage_ros ~/catkin_ws/src

Everything else is the same as the sphero_formation package. To change the RL algorithm, change to the appropriate script in reynolds_sim.launch. Also, change to the appropriate directories in order to save training results.

Results

Simulation video for flocking using DDPG can be found here and real Sphero robot here.