Run launch_lcbench_simulated.py, get ValueError exception about syne-tune

[geoalgo]

Hi @jjiantong
It seems that there is a compatibility issue with the parquet library to load the data.
Could you let us know both how you installed syne-tune (for instance did you do pip install syne-tune[extra]?) and what output you get if you do pip freeze with the interpreter you use for the example?

[jjiantong]

Thanks @geoalgo .
I installed syne-tune from source by

python3 -m venv st_venv
. st_venv/bin/activate
pip install wheel
pip install --upgrade pip
pip install -e '.[extra]'

Yes I did pip install -e '.[extra]'. The output of pip freeze was:

about-time==4.2.1
aiosignal==1.3.1
alabaster==0.7.13
alive-progress==3.1.0
attrs==22.2.0
autograd==1.5
Babel==2.12.1
black==22.3.0
boto3==1.26.101
botocore==1.29.101
botorch==0.8.3
certifi==2022.12.7
charset-normalizer==3.1.0
click==8.1.3
cma==3.2.2
cmake==3.26.1
coloredlogs==15.0.1
ConfigSpace==0.6.1
contextlib2==21.6.0
contourpy==1.0.7
coverage==7.2.2
cramjam==2.6.2
cycler==0.11.0
Deprecated==1.2.13
dill==0.3.6
distlib==0.3.6
docutils==0.18.1
exceptiongroup==1.1.1
fastparquet==2023.2.0
filelock==3.10.7
flake8==6.0.0
flatbuffers==23.3.3
fonttools==4.39.3
frozenlist==1.3.3
fsspec==2023.3.0
future==0.18.3
google-pasta==0.2.0
gpytorch==1.9.1
grapheme==0.6.0
grpcio==1.53.0
h5py==3.8.0
humanfriendly==10.0
idna==3.4
imagesize==1.4.1
importlib-metadata==4.13.0
importlib-resources==5.12.0
iniconfig==2.0.0
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.2.0
jsonschema==4.17.3
kiwisolver==1.4.4
latexcodec==2.0.1
linear-operator==0.3.0
lit==16.0.0
markdown-it-py==2.2.0
MarkupSafe==2.1.2
matplotlib==3.7.1
mccabe==0.7.0
mdit-py-plugins==0.3.5
mdurl==0.1.2
more-itertools==9.1.0
mpmath==1.3.0
msgpack==1.0.5
multipledispatch==0.6.0
multiprocess==0.70.14
mypy-extensions==1.0.0
myst-parser==1.0.0
networkx==3.0
numpy==1.23.5
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
onnxruntime==1.14.1
opt-einsum==3.3.0
packaging==23.0
pandas==1.5.3
pathos==0.3.0
pathspec==0.11.1
patsy==0.5.3
Pillow==9.4.0
platformdirs==3.2.0
pluggy==1.0.0
pox==0.3.2
ppft==1.7.6.6
protobuf==3.20.3
protobuf3-to-dict==0.1.5
pyaml==21.10.1
pybtex==0.24.0
pybtex-docutils==1.0.2
pycodestyle==2.10.0
pyflakes==3.0.1
Pygments==2.14.0
pymoo==0.6.0.1
pyparsing==3.0.9
pyro-api==0.1.2
pyro-ppl==1.8.4
pyrsistent==0.19.3
pytest==7.2.2
pytest-cov==4.0.0
pytest-timeout==2.1.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0
ray==2.3.1
requests==2.28.2
s3fs==0.4.2
s3transfer==0.6.0
sagemaker==2.142.0
schema==0.7.5
scikit-learn==1.2.2
scikit-optimize==0.9.0
scipy==1.10.1
six==1.16.0
smdebug-rulesconfig==1.0.1
snowballstemmer==2.2.0
sortedcontainers==2.4.0
Sphinx==6.1.3
sphinx-copybutton==0.5.1
sphinx-rtd-theme==1.2.0
sphinx_autodoc_typehints==1.22
sphinxcontrib-applehelp==1.0.4
sphinxcontrib-bibtex==2.5.0
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.1
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
statsmodels==0.13.5
sympy==1.11.1
-e git+https://github.com/awslabs/syne-tune.git@a711ac9e13e624fb230aa2fd28339ef895a4296a#egg=syne_tune
tabulate==0.9.0
tensorboardX==2.6
threadpoolctl==3.1.0
tomli==2.0.1
torch==2.0.0
torchvision==0.15.1
tqdm==4.65.0
triton==2.0.0
typing_extensions==4.5.0
ujson==5.7.0
urllib3==1.26.15
virtualenv==20.21.0
wrapt==1.15.0
xgboost==1.7.4
yahpo-gym==1.0.1
zipp==3.15.0

[geoalgo]

Thanks for sharing those details.
I checked and could reproduce the error in a fresh environment.
Is not related to fastparquet but to some issue with the example.

Here is a code that works to simulate lcbench with BO unblock you so that you dont need to wait a fix:

import logging
from syne_tune import StoppingCriterion, Tuner
from syne_tune.backend.simulator_backend.simulator_callback import SimulatorCallback
from syne_tune.blackbox_repository import BlackboxRepositoryBackend
from syne_tune.optimizer.baselines import BayesianOptimization

if __name__ == "__main__":
    logging.getLogger().setLevel(logging.INFO)

    trial_backend = BlackboxRepositoryBackend(
        blackbox_name="lcbench",
        dataset="airlines",
        elapsed_time_attr="time",
        surrogate="KNeighborsRegressor",  # 1-nn surrogate
        surrogate_kwargs={"n_neighbors": 1},
    )
    scheduler = BayesianOptimization(
        config_space=trial_backend.blackbox.configuration_space,
        max_resource_attr="epochs",
        mode="max",
        metric="val_accuracy",
        random_seed=0,
    )

    stop_criterion = StoppingCriterion(max_wallclock_time=7200)

    tuner = Tuner(
        trial_backend=trial_backend,
        scheduler=scheduler,
        stop_criterion=stop_criterion,
        n_workers=4,
        sleep_time=0,
        # This callback is required in order to make things work with the
        # simulator callback. It makes sure that results are stored with
        # simulated time (rather than real time), and that the time_keeper
        # is advanced properly whenever the tuner loop sleeps
        callbacks=[SimulatorCallback()],
    )
    tuner.run()

[mseeger]

Cool David, could you submit a fix for the example?

[jjiantong]

Thanks for your prompt answer. Unfortunately it didn't work on my side. I got the same problem as before. Did it work in your tested fresh environment?
But it's ok, I will look through the code and I can wait for a fix. Thank you again ;p

[geoalgo]

My bad, it was due to parquet version, sorry for the confusion.

Can you let us know if using this fastparquet version fixes the issue:

pip install fastparquet==0.8.1

it fixes it in my case.

[jjiantong]

Thanks! Changing the fastparquet version fixed the issue.