Is it possible to "suggest" just a configuration out of a fixed list? about hebo

[Alaya-in-Matrix]

We don't have a special API for that, but it's possible in HEBO with the observe function. you just need to feed the configurations with observe.

For example, let's say you want to minimize f(x) = x^2, and you want to provide two initial guess x = 3 and x = -5, then you can do it by the following code

import pandas as pd
from hebo.design_space import DesignSpace
from hebo.optimizers.hebo import HEBO

def obj_func(param):
    return (param['x'].values**2).reshape(-1, 1)

space = DesignSpace().parse([{'name' : 'x', 'type' : 'num', 'lb': -10, 'ub' : 10}])
opt   = HEBO(space)

initial_suggest = pd.DataFrame([{'x' : 3}, {'x' : -5}])
opt.observe(initial_suggest, obj_func(initial_suggest))

for i in range(3):
    rec = opt.suggest()
    opt.observe(rec, obj_func(rec))

print(opt.X)

[phyzhenli]

It is not enough with observe only. There must be suggest before observe.

The code should be modified as follows：

......
initial_suggest = pd.DataFrame([{'x' : 3}, {'x' : -5}])
initial_suggest = opt.suggest(n_suggestions=1, fix_input = initial_suggest)
opt.observe(initial_suggest, obj_func(initial_suggest))
......

[sebastianpinedaar]

Thank you for the reply.

I actually mean a set of fixed configurations to observe, not only initial configurations.

For instance, if we want to optimize "learning rate" from an arbitrary list : [1e-5, 3e-5, 6e-5, 7e-5, 4e-4, 5e-2, 8e-1],
how can I do it? I wonder because this list does not match any pattern from the possible design spaces.

Best,
Sebastian

[AntGro]

Hello Sebastian,

For now I think the best option would consist in considering "lr" as an integer parameter with lower bound 1 and upper bound the size of your list L (so for L = [1e-5, 3e-5, 6e-5, 7e-5, 4e-4, 5e-2, 8e-1] it would be 6) and to make your blackbox convert the suggestion lr = i to lr = L[i]. Note that you could also use a categorical parameter to directly suggest an element of your list, but then HEBO would not be aware of the order relations among the "categories".