miraclemarvel55 / chatglm-rlhf Goto Github PK

你好，训练后的模型没有保存下来呢，代码中没有保存模型的部分啊，这怎么检验呢

出现这个错误了，大佬有解吗？
['/home/zhangshuhao/GLM_RLHF/ChatGLM-RLHF', '/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python38.zip', '/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8', '/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/lib-dynload', '/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages', '/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/trl-0.4.2.dev0-py3.8.egg']
The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████| 8/8 [00:07<00:00, 1.09it/s]
Some weights of the model checkpoint at THUDM/chatglm-6b were not used when initializing ChatGLMModel: ['lm_head.weight']

• This IS expected if you are initializing ChatGLMModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing ChatGLMModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
  The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
  Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████| 8/8 [00:07<00:00, 1.06it/s]
  0%| | 0/16 [00:00<?, ?it/s]The dtype of attention mask (torch.int64) is not bool
  你的主人是谁？
  ['作为一个人工智能助手,我没有真正的主人。']
  Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
  0%| | 0/16 [00:01<?, ?it/s]
  Traceback (most recent call last):
  File "chatglm_rlhf.py", line 212, in
  main(prompts_path = dialogues_path)
  File "chatglm_rlhf.py", line 164, in main
  reward = reward_model(gen_texts=gen_texts, good_answers=good_answers, bad_answers=bad_answers).unsqueeze(1)
  File "/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  return forward_call(*args, **kwargs)
  File "/home/zhangshuhao/GLM_RLHF/ChatGLM-RLHF/models_rlhf.py", line 121, in forward
  jaccards = torch.tensor(np.vectorize(jaccard_s1)(ids[-len(examples):]), dtype=coses.dtype, device=coses.device)
  File "/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2329, in call
  return self._vectorize_call(func=func, args=vargs)
  File "/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2407, in _vectorize_call
  ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)
  File "/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2361, in _get_ufunc_and_otypes
  args = [asarray(arg) for arg in args]
  File "/home/zhangshuhao/anaconda3/envs/ChatGLM-RLHF/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2361, in
  args = [asarray(arg) for arg in args]
  ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5,) + inhomogeneous part.

能够更新一下吗

作者你好，疑问如题

When I use numpy 1.24.x, I got error in jaccards = torch.tensor(np.vectorize(jaccard_s1)(ids[-len(examples):]), dtype=coses.dtype, device=coses.device)
error as follwing:
setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.

but I use 1.22.2, I got no error.