befelix / safeopt

Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization

This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also provides a more scalable implementation based on [3] as well as an implementation for the original algorithm in [4]. The code can be used to automatically optimize a performance measures subject to a safety constraint by adapting parameters. The prefered way of citing this code is by referring to [1] or [2].

[1]	F. Berkenkamp, A. P. Schoellig, A. Krause, Safe Controller Optimization for Quadrotors with Gaussian Processes in Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 491-496.

[2]	F. Berkenkamp, A. Krause, A. P. Schoellig, Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics, ArXiv, 2016, arXiv:1602.04450 [cs.RO].

[3]	Rikky R.P.R. Duivenvoorden, Felix Berkenkamp, Nicolas Carion, Andreas Krause, Angela P. Schoellig, Constrained Bayesian optimization with Particle Swarms for Safe Adaptive Controller Tuning, in Proc. of the IFAC (International Federation of Automatic Control) World Congress, 2017.

[4]	Y. Sui, A. Gotovos, J. W. Burdick, and A. Krause, Safe exploration for optimization with Gaussian processes in Proc. of the International Conference on Machine Learning (ICML), 2015, pp. 997–1005.

Warning: Maintenance mode

This package is no longer actively maintained. That bein said, pull requests to add functionality or fix bugs are always welcome.

Installation

The easiest way to install the necessary python libraries is by installing pip (e.g. apt-get install python-pip on Ubuntu) and running

pip install safeopt

Alternatively you can clone the repository and install it using

python setup.py install

Usage

The easiest way to get familiar with the library is to run the interactive example ipython notebooks!

Make sure that the ipywidgets module is installed. All functions and classes are documented on Read The Docs.

License

The code is licenced under the MIT license and free to use by anyone without any restrictions.

safeopt's Issues

Effect of search space boundaries on exploration

(Only boundaries for GP changed in the 2 plots)
Do you have any hints how a change in the boundaries effect exploration that much?
Our understanding was, that boundaries shouldn't effect exploration at all but only act as a limit.

upper bound constraints

Hi. I wonder whether it is possible to add the upper bound constraints, i.e., gps<=fmax, which are opposite to the lower bound constraints, i.e., gps>=fmin. It seems to be possible to add the upper bound constraints to the "comput_safe_set" functioon when lipschitz constant L is infinite.

2d-optimization flipped axis

we have used this toolbox as an example in our application (https://github.com/upb-lea/openmodelica-microgrid-gym), where we will design a current controller (standard PI; Kp/Ki) for an inverter.

expected behavoiur

When fliping the dimensions of our two dimensional optimization space, we would expect the ploted results should only flip the axis.

Observed behavior

However, when observing the point marked in green, the values change!

what we investigated

bounds are flipped accourdingly

Ideas

We have the impression, that the first dimension is explored much better than the second dimension. in the left plot the y values never exeed 0.06 even though they are deemed save in the right experiment. Similarly in the right plot, it never dares to try above 150, but in the left it is explored and also save. Therefore we assume it is related to the handling of expanders.

Do you have any hints on how to investigate it further. We are not certain that it is a bug, but the anti-symmetry seems odd.

plot_3d=True

Hi. Your algorithm is pretty good and it's very useful in many cases.
But,there is an error:"TypeError: object of type 'numpy.float64' has no len()." when i try to change plot_3d=False to True in 2d_example.n
It's not a big matter in Jupyter notebook(In jupyter it can still display the 3D function images),but it is unexecutable in Pycham.
Do you know how to make it?
Thanks.

MovieLen 100k with the original SafeOpt algorithm

Hi there!

I'm trying to replicate the result of SafeOpt on the MovieLen 100k dataset and use it as a baseline for my class project. I tried to follow the procedure described in the paper but I failed to reproduce the result. Can I get some help? Thank you!

Here are the steps that I followed:

And here is my code:

The problem right now is that

• the algorithm breaks the safety constraint pretty quick (explored movies that have a 1.0 rating)
• the algorithm tends to be stuck at one or multiple high rating recommendations after some iterations and it is not able to generate novel results.
• similar to the second point, it explores only a small portion of the reachable set

We experimented with different std and noise variance but did not actually make significant improvement. I wonder do you notice any mistake we made in our code? In addition, would it be possible to share the code you used for the evaluation in the paper?

Thank you very much!