Comments (6)
1.安装依赖后,执行命令
streamlit run web_feadback.py --server.port=8080
,报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 2023-04-26 11:11:07.907 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 15, in <module> from modeling_chatglm import ChatGLMForConditionalGeneration ModuleNotFoundError: No module named 'modeling_chatglm'
2.将项目根目录的
modeling_chatglm.py
、configuration_chatglm.py
文件拷贝到 web_ui 目录下,再次执行streamlit run web_feadback.py --server.port=8080
,再次报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00, 1.16it/s] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 11:15:21.961 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2198, in greedy_search model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs) File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 988, in prepare_inputs_for_generation mask_position = seq.index(mask_token) ValueError: 150001 is not in list
3.将步骤2中添加的
modeling_chatglm.py
、configuration_chatglm.py
文件删除,替换为THUDM/ChatGLM-6B
项目chatglm-6b
模型中提供的文件,仍旧报错:/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00, 1.15it/s] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 11:32:12.915 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search model_kwargs = self._update_model_kwargs_for_generation( File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation attention_mask = torch.cat( IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
把ChatGLMForConditionalGeneration 换成AutoModel 把token也换成AutoToken
清华把他们上传HF的模型全部换过了
from chatglm_lora_multi-gpu.
1.将web_feadback.py
中 ChatGLMForConditionalGeneration 修改为 AutoModel
if 'model' not in st.session_state:
with st.spinner('Loading Model...'):
#config = PeftConfig.from_pretrained(peft_model_path)
peft_path = "chatglm-lora-new.pt"
peft_config = LoraConfig(
task_type=TaskType.CAUSAL_LM, inference_mode=True,
r=8,
lora_alpha=32,
lora_dropout=0.1)
model = AutoModel.from_pretrained(
"/models/chatglm-6b",
cache_dir ='./',
trust_remote_code=True
).float().to(device)
model = get_peft_model(model, peft_config)
model.load_state_dict(torch.load(peft_path), strict=False)
'''model = PeftModel.from_pretrained(
model,
peft_model_path
)'''
tokenizer = AutoTokenizer.from_pretrained(
"/models/chatglm-6b",
cache_dir ='./',
trust_remote_code=True
)
st.session_state['model'] = model
st.session_state['tokenizer'] = tokenizer
2.再次执行,出现以下错误,是还有其他地方要改吗? @liangwq
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 14:26:02.449 Uncaught app exception
Traceback (most recent call last):
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module>
main()
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main
start_evaluate_page()
File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page
out = model.generate(
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
outputs = self.base_model.generate(**kwargs)
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
return self.greedy_search(
File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search
model_kwargs = self._update_model_kwargs_for_generation(
File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation
attention_mask = torch.cat(
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
from chatglm_lora_multi-gpu.
1.将
web_feadback.py
中 ChatGLMForConditionalGeneration 修改为 AutoModelif 'model' not in st.session_state: with st.spinner('Loading Model...'): #config = PeftConfig.from_pretrained(peft_model_path) peft_path = "chatglm-lora-new.pt" peft_config = LoraConfig( task_type=TaskType.CAUSAL_LM, inference_mode=True, r=8, lora_alpha=32, lora_dropout=0.1) model = AutoModel.from_pretrained( "/models/chatglm-6b", cache_dir ='./', trust_remote_code=True ).float().to(device) model = get_peft_model(model, peft_config) model.load_state_dict(torch.load(peft_path), strict=False) '''model = PeftModel.from_pretrained( model, peft_model_path )''' tokenizer = AutoTokenizer.from_pretrained( "/models/chatglm-6b", cache_dir ='./', trust_remote_code=True ) st.session_state['model'] = model st.session_state['tokenizer'] = tokenizer
2.再次执行,出现以下错误,是还有其他地方要改吗? @liangwq
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths... warn(msg) /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 6.1 CUDA SETUP: Detected CUDA version 114 /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! warn(msg) CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so... 标注数据集已创建。 Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it] Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. 2023-04-26 14:26:02.449 Uncaught app exception Traceback (most recent call last): File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module> main() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main start_evaluate_page() File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page out = model.generate( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate outputs = self.base_model.generate(**kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate return self.greedy_search( File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search model_kwargs = self._update_model_kwargs_for_generation( File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation attention_mask = torch.cat( IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)
把你前面下载的chatglm模型删除,重新让hf下载
from chatglm_lora_multi-gpu.
我的chatglm-6b模型就是最新的版本,还是有这个问题 @liangwq
from chatglm_lora_multi-gpu.
我的chatglm-6b模型就是最新的版本,还是有这个问题 @liangwq
116行就是:batch = st.session_state['tokenizer'](input_text, return_tensors="pt")
这部分就是利用模型对你输入做预测
这样你分别试试:
1.把lora去掉看看能不能跑通
2.看看tokenizier是不是 最新的
from chatglm_lora_multi-gpu.
我本地测试没有这个问题
from chatglm_lora_multi-gpu.
Related Issues (20)
- 推理阶段deepspeed --num_gpus 2 chatglm_multi_gpu_inference.py报错 HOT 3
- 运行deepspeed --num_gpus 2 chatglm_deepspeed_inference.py进行推理,没有生成yitu_output.csv是什么原因呢? HOT 3
- 如何训练自己数据集 HOT 1
- 显存占用问题 HOT 2
- 你的README.md与Chatglm_lora_multi-gpu/data HOT 1
- 运行web_ui.py,报错:NameError: name 'LoraConfig' is not defined HOT 1
- deepspeed和lora HOT 4
- 模型是否存在信息泄露 HOT 4
- alps import error HOT 1
- 多卡并行问题 HOT 1
- chatglm用deepspeed多卡推理问题 HOT 1
- chatglm做图应用怎么使用 HOT 1
- 多卡deepspeed模式 HOT 2
- Deepspeed并未生效 HOT 1
- 推理问题 HOT 17
- langchain版本是多少 HOT 1
- huggingface_hub.utils._validators.HFValidationError HOT 3
- 解析相应报错 HOT 3
- 看下我的搜索结果对不对
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chatglm_lora_multi-gpu.