crash in VLLM about openchat

[Valdanitooooo]

I get another error:

2023-07-18 15:53:21,350	ERROR services.py:1207 -- Failed to start the dashboard , return code 1
2023-07-18 15:53:21,350	ERROR services.py:1232 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2023-07-18 15:53:21,350	ERROR services.py:1276 -- 
The last 20 lines of /tmp/ray/session_2023-07-18_15-53-19_820841_46100/logs/dashboard.log (it contains the error message from the dashboard): 
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/dashboard/modules/log/log_manager.py", line 8, in <module>
    from ray.util.state.common import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/__init__.py", line 1, in <module>
    from ray.util.state.api import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/api.py", line 17, in <module>
    from ray.util.state.common import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/common.py", line 120, in <module>
    @dataclass(init=True)
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/pydantic/dataclasses.py", line 139, in dataclass
    assert init is False, 'pydantic.dataclasses.dataclass only supports init=False'
AssertionError: pydantic.dataclasses.dataclass only supports init=False
2023-07-18 15:53:21,467	INFO worker.py:1636 -- Started a local Ray instance.
[2023-07-18 15:53:22,307 E 46100 46100] core_worker.cc:193: Failed to register worker 01000000ffffffffffffffffffffffffffffffffffffffffffffffff to Raylet. IOError: [RayletClient] Unable to register worker with raylet. No such file or directory

[imoneoi]

Have you fixed the issue? We released a new version and tested the following setup:

conda create -y --name openchat
conda activate openchat

conda install -y python
conda install -y cudatoolkit-dev -c conda-forge
pip3 install torch torchvision torchaudio

pip3 install packaging ninja
pip3 install --no-build-isolation "flash-attn<2"

pip3 install ochat

[butterluo]

I get another error:

2023-07-18 15:53:21,350	ERROR services.py:1207 -- Failed to start the dashboard , return code 1
2023-07-18 15:53:21,350	ERROR services.py:1232 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2023-07-18 15:53:21,350	ERROR services.py:1276 -- 
The last 20 lines of /tmp/ray/session_2023-07-18_15-53-19_820841_46100/logs/dashboard.log (it contains the error message from the dashboard): 
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/dashboard/modules/log/log_manager.py", line 8, in <module>
    from ray.util.state.common import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/__init__.py", line 1, in <module>
    from ray.util.state.api import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/api.py", line 17, in <module>
    from ray.util.state.common import (
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/ray/util/state/common.py", line 120, in <module>
    @dataclass(init=True)
  File "/gai_data/anaconda3/envs/fastchat/lib/python3.10/site-packages/pydantic/dataclasses.py", line 139, in dataclass
    assert init is False, 'pydantic.dataclasses.dataclass only supports init=False'
AssertionError: pydantic.dataclasses.dataclass only supports init=False
2023-07-18 15:53:21,467	INFO worker.py:1636 -- Started a local Ray instance.
[2023-07-18 15:53:22,307 E 46100 46100] core_worker.cc:193: Failed to register worker 01000000ffffffffffffffffffffffffffffffffffffffffffffffff to Raylet. IOError: [RayletClient] Unable to register worker with raylet. No such file or directory

Any update for this issue? I met this problem too.

[timothylimyl]

@imoneoi given that OpenChat adds a special token, the same changes has to be made in vllm right? I believe vllm uses the default hugging face model and tokenizer.

Did OpenChat integrate with vllm?