The results of the suggest method is not returned in preferential optimization with 2 type of variable. about optuna-dashboard

[c-bata]

Thank you for reporting! @contramundum53 Could you take a look if you have time?

[contramundum53]

Oh I see, are you using Windows?
(or possibly some 32-bit environment?)
This seems to be an integer overflow, and somehow your numpy seems to be using np.int32 as default.

[contramundum53]

I updated the PR to use math.prod instead of np.prod so that the integer won't overflow. Does that fix your problem?

[hrntsm]

I use windows.
I tried the updated PR and it works fine.
Thank you very much!

[contramundum53]

I couldn't reproduce the problem.

from optuna_dashboard.preferential import create_study
from optuna_dashboard.preferential.samplers.gp import PreferentialGPSampler

study = create_study(
    n_generate=4,
    sampler=PreferentialGPSampler(),
    load_if_exists=True,
)

while True:
    assert study.should_generate()

    trial = study.ask()
    print(trial.number)
    # 1. Ask new parameters
    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)
    _ = trial.suggest_float("x", 0, 255)  

    if len(study.trials) >= 4:
        study.report_preference(study.trials[:3], study.trials[-1])

This worked perfectly. Could you try this script?

[hrntsm]

@contramundum53

Thank you for your response. When I run your code, it still stops.
The following is the output for reference.

[I 2024-02-06 19:17:49,078] A new study created in memory with name: no-name-d4dc0578-7d44-47f0-b9fd-a7bc9e3c9cba
C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna_dashboard\preferential\_study.py:297: FutureWarning: system_attrs has been deprecated in v3.1.0. This feature will be removed in v5.0.0. See https://github.com/optuna/optuna/releases/tag/v3.1.0.
  return len(active_trials) < get_n_generate(self._study.system_attrs)
0
C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna_dashboard\preferential\samplers\gp.py:341: FutureWarning: system_attrs has been deprecated in v3.1.0. This feature will be removed in v5.0.0. See https://github.com/optuna/optuna/releases/tag/v3.1.0.
  preferences = get_preferences(study.system_attrs)
C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna_dashboard\preferential\samplers\gp.py:467: UserWarning: Dynamic search space detected. Falling back to RandomSampler.
  warnings.warn(
1
2
3
4

It doesn't work, so the output when KeyboardInterrupt is done is as follows.

Traceback (most recent call last):
  File "c:\Users\dev\Desktop\potest\new.py", line 16, in <module>
    r = trial.suggest_int("r", 0, 255)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna\_convert_positional_args.py", line 83, in converter_wrapper
    return func(**kwargs)
           ^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna\trial\_trial.py", line 326, in suggest_int
    suggested_value = int(self._suggest(name, distribution))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna\trial\_trial.py", line 631, in _suggest
    elif self._is_relative_param(name, distribution):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna\trial\_trial.py", line 663, in _is_relative_param
    if name not in self.relative_params:
                   ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna\trial\_trial.py", line 73, in relative_params
    self._relative_params = self.study.sampler.sample_relative(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna_dashboard\preferential\samplers\gp.py", line 439, in sample_relative
    np.array([trans.transform(dict(params)) for params in all_param_combinations]),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dev\Desktop\potest\.venv\Lib\site-packages\optuna_dashboard\preferential\samplers\gp.py", line 439, in <listcomp>
    np.array([trans.transform(dict(params)) for params in all_param_combinations]),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

I confirmed that the following three will work without problems.

    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)

This problem occurred on my Windows PC. So I checked if the same problem occurs on macs, and it works without any problem on mac.

[hrntsm]

To check, I changed the float to int as shown below, but this did not work either.

from optuna_dashboard.preferential import create_study
from optuna_dashboard.preferential.samplers.gp import PreferentialGPSampler

study = create_study(
    n_generate=4,
    sampler=PreferentialGPSampler(),
    load_if_exists=True,
)

while True:
    assert study.should_generate()

    trial = study.ask()
    print(trial.number)
    # 1. Ask new parameters
    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)
    _ = trial.suggest_int("x", 0, 255)

    if len(study.trials) >= 4:
        study.report_preference(study.trials[:3], study.trials[-1])

However the following(5 suggest method) worked. Sorry I am not sure what is causing the problem.

    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)
    _ = trial.suggest_int("x", 0, 255)
    _ = trial.suggest_int("y", 0, 255)

[contramundum53]

@hrntsm I found a bug in our implementation, and fixed it in #797. Could you check whether the problem fixes?

[hrntsm]

@contramundum53 I checked but that did not solve the problem.

Based on the PR, I tried to find out the cause. As a result, the following part has a negative value.
It seems strange that the name "search_space_size" should be negative, is this correct?

optuna-dashboard/optuna_dashboard/preferential/samplers/gp.py

Line 404 in cfadf70

search_space_size = np.prod(

Therefore, the following IF is True, which seems to be the cause of the huge number of searches.

optuna-dashboard/optuna_dashboard/preferential/samplers/gp.py

Line 432 in cfadf70

if is_all_discrete and can_evaluate_all:

[hrntsm]

In this case, there are four 0-255, so the result is 4228,250,625(255x255x255x255), but for some reason it was a negative value.

search_space_size = np.prod([255,255,255,255])
# search_space_size is -66716671

[contramundum53]

And if you change the fourth parameter to suggest_float, would it solve your problem?