Experience with 4D or higher parameter spaces about bayesianoptimization

[till-m]

Hi @BSchuett-fzi,

I assume that your function is reasonably well conditioned and differentiable. If that's the case, 2000 iterations really should not happen.
One quick and easy thing to try: By default, this optimizer uses the Matern kernel with an isotropic lengthscale setting. If your problem is not isotropic, I could see this causing issues. You can use

# instatiate the optimizer
...
# replace kernel
kernel = Matern(nu=2.5),
            alpha=1e-6,
            normalize_y=True,
            n_restarts_optimizer=5,
            length_scale=np.ones(4), # default = 1.0
            random_state=optimizer.random_state
)
optimizer.set_gp_params(kernel=kernel)
# optimization loop
...

to replace it with an anisotropic one.

Let us know if this helps.

[BSchuett-fzi]

Thanks for your hint.
I'm sorry to ask, but do I still use a GaussianProcessRegressor? I have this code now:

gp = GaussianProcessRegressor(kernel = Matern(nu=2.5),
                            alpha=1e-6,
                            normalize_y=True,
                            n_restarts_optimizer=5,
                            length_scale=np.ones(4),  # default = 1.0
                            random_state=optimizer._random_state)
            optimizer.set_gp_params(kernel=gp)

But the length_scale is not an existing argument. I use scikit-learn 1.3.2.

[till-m]

Hi @BSchuett-fzi,

no worries. Yes, it still uses a gp. the .set_gp_params function simply replaces some parts of the gp, in this case the kernel. The length scale is an argument of the kernel, not an argument of the gp (hence your code not working).

I hope this helps, please let us know otherwise :)

[till-m]

Hi @BSchuett-fzi,

I will close this issue for now with the hope that the problem has been fixed. If it persits, feel free to reopen.