Inference without Deepspeed about deepspeed

[mrwyattii]

Hi @aaronnat23 the error 'deepspeed' is not recognized as an internal or external command indicates that the deepspeed launcher script is not on your system path. When you install in a conda environment, there should be a bin/deepspeed installed as well. Can you please share the steps you used to setup your conda environment and install deepspeed?

[aaronnat23]

HI @mrwyattii thanks for the reply,

setup for conda environment
conda create -n moe_llava python=3.10
conda activate moe_llava
pip install --upgrade pip
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

setup for deepspeed
#5152 from windows installation
git clone repo
Launch cmd console with Administrator privilege for creating required symlink folders
set DS_BUILD_OPS=0
set DS_BUILD_SPARSE_ATTN=0
set DS_BUILD_EVOFORMER_ATTN=0
Run python setup.py bdist_wheel to build wheel in dist folder
pip install deepspeed-0.13.3+b20c4674-py3-none-any.whl from dist folder

i notice that in my conda env C:\Users\E\anaconda3\envs\moe_llava\bin, there is no deepspeed.

However in C:\Users\E\anaconda3\envs\moe_llava\Lib\site-packages and in "pip freeze" i noticed that it is already installed

how can i add it to system path or what else do i need to do