krishanrana / bcf

[NeurIPS 2021 DDM Workshop] A hybrid control strategy for combining deep RL and classical robotic controllers

Bayesian Controller Fusion

A hybrid control strategy for combining deep RL and classical robotic controllers. We provide two environments for both navigation and reaching tasks. For each task, we additionally provide traditional handcrafted controllers that can solve part of the task, however are not the optimal solution. Visit our project page for more information and video demonstrations.

Installation

The only requirement is that you have Conda installed on your system, and NVIDIA drivers installed if you want CUDA acceleration. To install all the required python dependencies run the following command within conda.

git clone https://github.com/krishanrana/bcf.git
cd bcf
conda env create --file env_requirements.yml/

For the manipuliabity maximising reacher task, you will additionally require PyRep and CoppeliaSim. Head to the PyRep github page for installation instructions for this environment.

Usage

The complete training pipeline for BCF and the baselines compared in this work are provides in 'main.py'. The file allows for several input arguments to allow the user to specify the task, algorithm, prior controller, and the respective hyperparameters.

python3 main.py --task "navigation" --method "BCF" --prior_controller "APF" --sigma_prior 0.4 --num_agents 10

Logging

All results are logged using Weights and Biases. An account and initial login is required to initialise logging as described on thier website.

Citation

  @article{rana2021bayesian,
    title={Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Robotics},
    author={Rana, Krishan and Dasagi, Vibhavari and Haviland, Jesse and Talbot, Ben and Milford, Michael and S{\"u}nderhauf, Niko},
    journal={arXiv preprint arXiv:2107.09822},
    year={2021}
  }

bcf's Issues

Can't reproduce the results of paper.

I find something strange in the results of the code. My results of navigation are as follows. The hyper-parameters are same with yours (Num_Agents =5).

Besides, I have a question for the BCF. I see your only evaluate the ensemble policy during training. Do we need the prior controller to do evaluation ? Or just use the ensemble policy to give action?