hejia-zhang / awesome-model-based-reinforcement-learning

A curated list of awesome Model-based reinforcement learning resources

Awesome Model-based Reinforcement Learning

A curated list of awesome Model-based Reinforcement Learning resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search

Model-based Reinforcement Learning is gaining popularity in Robotics community. These are some of the awesome resources!

Contributing

Please help contribute this list by adding pull request

Markdown format:

- Paper Name [[pdf]](link) [[code]](link)
  - Author 1, Author 2 and Author 3. *Conference/Journal Year*

Table of Contents

• Thesis
• Survey
• Conference Papers
• Journal Papers
• Tutorials
• Tools

Thesis

• Efficient Reinforcement Learning using Gaussian Processes. [pdf]
  • Marc Peter Deisenroth.

Survey

2017

• Survey of Model-Based Reinforcement Learning: Applications on Robotics. [pdf]
  • Athanasios S. Polydoros and Lazaros Nalpantidis. J Intell Robot Syst 2017

CV/CG

• Curiosity-driven Exploration by Self-supervised Prediction. [pdf] [code]
  • Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell. ICML 2017

Conference Papers

Physics Model

• Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning. [pdf] [code]
  • Marc Toussaint, Kelsey R. Allen, Kevin A. Smith, Joshua B. Tenenbaum. RSS 2018
• A convex, smooth and invertible contact model for trajectory optimization. [pdf]
  • Emanuel Todorov. ICRA 2011
• A Modular Differentiable Rigid Body Physics Engine. [pdf] [code]
  • Filipe de Avila Belbute-Peres, J. Zico Kolter. Deep Reinforcement Learning Symposium, NIPS 2017
• A DIFFERENTIABLE PHYSICS ENGINE FOR DEEP LEARNING IN ROBOTICS. [pdf]
  • Jonas Degrave, Michiel Hermans, Joni Dambre, Francis wyffels. ICLR 2017
• Discovery of Complex Behaviors through Contact-Invariant Optimization. [pdf]
  • Igor Mordatch, Emanuel Tordorov, Zoran Popovic. TOG'12

Hybrid model-based and model-free algorithm

• Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning. [pdf] [code]
  • Anusha Nagabandi, Gregory Kahn, Ronald S. Fearing, Sergey Levine.
• Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning. [pdf]
  • Yevgen Chebotar, Karol Hausman, Marvin Zhang, Gaurav Sukhatme, Stefan Schaal, Sergey Levine. ICML 2017

Optimal Control

• Local Gaussian Process Regression for Real-time Model-based Robot Control. [pdf]
  • Duy Nguyen-Tuong and Jan Peters. IROS 2008

Local model

• Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics. [pdf] [code]
  • Sergey Levine and Pieter Abbeel. NIPS 2014

Learn in latent space

Foward Dynamics Model

• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images. [pdf] [code]
  • Manuel Watter, Jost Tobias Springenberg, Martin Riedmiller, Joschka Boedecker. ICRA 2017
• Deep Spatial Autoencoders for Visuomotor Learning. [pdf] [code]
  • Chelsea Finn, Xin Yu Tan, Yan Duan, Trevor Darrell, Sergey Levine, Pieter Abbeel. ICRA 2016

Gaussian Process

• Data-Efficient Reinforcement Learning in Continuous-State POMDPs. [pdf]
  • Rowan McAllister, Carl Rasmussen. NIPS 2017
• Improving PILCO with Bayesian Neural Network Dynamics Models. [pdf]
  • Yarin Gal and Rowan Thomas McAllister and Carl Edward Rasmussen. Data-Efficient Machine Learning workshop, ICML, 2016
• PILCO: A Model-Based and Data-Efficient Approach to Policy Search. [pdf] [code] [unofficial code]
  • Marc Peter Deisenroth, Carl Rasmussen. ICML 2011
• Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning. [pdf]
  • Marc Peter Deisenroth, Carl Edward Rasmussen and Dieter Fox. RSS 2011
• Learning Dynamics Across Similar Spatiotemporally-Evolving Physical Systems. [pdf]
  • Joshua Whitman, Girish Chowdhary. CoRL 2017

Journal Papers

Tutorials

• Deep RL Bootcamp Lecture 9 Model-based Reinforcement Learning. Chelsea Finn (UC Berkeley) [link]
• Highlight Talk: Gaussian Processes for Data Efficient Learning. Marc Diesenroth [link]

Tools

• GPFlow
• GPy

License

To the extent possible under law, we have waived all copyright and related or neighboring rights to this work.