Coder Social home page Coder Social logo

Comments (6)

liangwq avatar liangwq commented on August 17, 2024

1.安装依赖后，执行命令 streamlit run web_feadback.py --server.port=8080，报错：

/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
2023-04-26 11:11:07.907 Uncaught app exception
Traceback (most recent call last):
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 15, in <module>
    from modeling_chatglm import ChatGLMForConditionalGeneration
ModuleNotFoundError: No module named 'modeling_chatglm'

2.将项目根目录的 modeling_chatglm.py、configuration_chatglm.py 文件拷贝到 web_ui 目录下，再次执行 streamlit run web_feadback.py --server.port=8080，再次报错：

/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00,  1.16it/s]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 11:15:21.961 Uncaught app exception
Traceback (most recent call last):
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module>
    main()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main
    start_evaluate_page()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page
    out = model.generate(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
    return self.greedy_search(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2198, in greedy_search
    model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
  File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 988, in prepare_inputs_for_generation
    mask_position = seq.index(mask_token)
ValueError: 150001 is not in list

3.将步骤2中添加的modeling_chatglm.py、configuration_chatglm.py 文件删除，替换为THUDM/ChatGLM-6B项目 chatglm-6b 模型中提供的文件，仍旧报错：

/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"'), PosixPath('history -a; history -a; printf "\\033]0;%s@%s')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 100%|█████████████████████████████████████████████| 8/8 [00:06<00:00,  1.15it/s]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 11:32:12.915 Uncaught app exception
Traceback (most recent call last):
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 289, in <module>
    main()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 282, in main
    start_evaluate_page()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 117, in start_evaluate_page
    out = model.generate(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
    return self.greedy_search(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search
    model_kwargs = self._update_model_kwargs_for_generation(
  File "/root/Chatglm_lora_multi-gpu/webui/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation
    attention_mask = torch.cat(
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)

把ChatGLMForConditionalGeneration 换成AutoModel 把token也换成AutoToken
清华把他们上传HF的模型全部换过了

from chatglm_lora_multi-gpu.

Cola-Ice avatar Cola-Ice commented on August 17, 2024

1.将web_feadback.py中 ChatGLMForConditionalGeneration 修改为 AutoModel

if 'model' not in st.session_state:
    with st.spinner('Loading Model...'):
        #config = PeftConfig.from_pretrained(peft_model_path)
        peft_path = "chatglm-lora-new.pt"

        peft_config = LoraConfig(
            task_type=TaskType.CAUSAL_LM, inference_mode=True,
            r=8,
            lora_alpha=32,
            lora_dropout=0.1)


        model = AutoModel.from_pretrained(
            "/models/chatglm-6b",
            cache_dir ='./',
            trust_remote_code=True
        ).float().to(device)
        model = get_peft_model(model, peft_config)
        model.load_state_dict(torch.load(peft_path), strict=False)
        '''model = PeftModel.from_pretrained(
            model, 
            peft_model_path
        )'''
        tokenizer = AutoTokenizer.from_pretrained(
            "/models/chatglm-6b",
            cache_dir ='./',
            trust_remote_code=True
        )
        st.session_state['model'] = model
        st.session_state['tokenizer'] = tokenizer

2.再次执行，出现以下错误，是还有其他地方要改吗？ @liangwq

/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 14:26:02.449 Uncaught app exception
Traceback (most recent call last):
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module>
    main()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main
    start_evaluate_page()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page
    out = model.generate(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
    return self.greedy_search(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search
    model_kwargs = self._update_model_kwargs_for_generation(
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation
    attention_mask = torch.cat(
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)

from chatglm_lora_multi-gpu.

liangwq avatar liangwq commented on August 17, 2024

1.将web_feadback.py中 ChatGLMForConditionalGeneration 修改为 AutoModel

if 'model' not in st.session_state:
    with st.spinner('Loading Model...'):
        #config = PeftConfig.from_pretrained(peft_model_path)
        peft_path = "chatglm-lora-new.pt"

        peft_config = LoraConfig(
            task_type=TaskType.CAUSAL_LM, inference_mode=True,
            r=8,
            lora_alpha=32,
            lora_dropout=0.1)


        model = AutoModel.from_pretrained(
            "/models/chatglm-6b",
            cache_dir ='./',
            trust_remote_code=True
        ).float().to(device)
        model = get_peft_model(model, peft_config)
        model.load_state_dict(torch.load(peft_path), strict=False)
        '''model = PeftModel.from_pretrained(
            model, 
            peft_model_path
        )'''
        tokenizer = AutoTokenizer.from_pretrained(
            "/models/chatglm-6b",
            cache_dir ='./',
            trust_remote_code=True
        )
        st.session_state['model'] = model
        st.session_state['tokenizer'] = tokenizer

2.再次执行，出现以下错误，是还有其他地方要改吗？ @liangwq

/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('history -a; history -a; printf "\\033]0;%s@%s'), PosixPath('%s\\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\\~}"')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 114
/root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so...
标注数据集已创建。
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 8/8 [01:29<00:00, 11.24s/it]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
2023-04-26 14:26:02.449 Uncaught app exception
Traceback (most recent call last):
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 288, in <module>
    main()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 281, in main
    start_evaluate_page()
  File "/root/Chatglm_lora_multi-gpu/webui/web_feadback.py", line 116, in start_evaluate_page
    out = model.generate(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/peft/peft_model.py", line 729, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1406, in generate
    return self.greedy_search(
  File "/root/anaconda3/envs/lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2245, in greedy_search
    model_kwargs = self._update_model_kwargs_for_generation(
  File "/root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py", line 1085, in _update_model_kwargs_for_generation
    attention_mask = torch.cat(
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 3)

把你前面下载的chatglm模型删除，重新让hf下载

from chatglm_lora_multi-gpu.

Cola-Ice avatar Cola-Ice commented on August 17, 2024

我的chatglm-6b模型就是最新的版本，还是有这个问题 @liangwq

from chatglm_lora_multi-gpu.

liangwq avatar liangwq commented on August 17, 2024

我的chatglm-6b模型就是最新的版本，还是有这个问题 @liangwq

116行就是：batch = st.session_state['tokenizer'](input_text, return_tensors="pt")
这部分就是利用模型对你输入做预测
这样你分别试试：
1.把lora去掉看看能不能跑通
2.看看tokenizier是不是 最新的

from chatglm_lora_multi-gpu.

liangwq avatar liangwq commented on August 17, 2024

我本地测试没有这个问题

from chatglm_lora_multi-gpu.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.