Comments (6)
https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/HumanEval/eval_instruct.py
你可以查看这个
from deepseek-coder.
tokenizer = AutoTokenizer.from_pretrained(
args.tuned_model,
padding_side="left",
trust_remote_code=True
)
tokenizer.pad_token = tokenizer.eos_token
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained(
args.tuned_model,
device_map="auto",
torch_dtype=torch.float16,
trust_remote_code=True,
pad_token_id=tokenizer.eos_token_id
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
outputs = model.generate(**inputs, max_new_tokens=128)
generated_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
我微调了一个代码分类模型,并使用上面的代码使用微调后的模型做预测,模型输出如下内容:
### Response: Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald Ald
测试的 prompt 为微调数据去除了 ### Response: 后的内容的形式。
我打印了inputs['input_ids']如下:
tensor([[32013, 2042, 417, 274, 20926, 14244, 20391, 13, 185, 13518,
3649, 3475, 25, 185, 7983, 498, 3192, 254, 1884, 2974,
5396, 16371, 7551, 13, 185, 13518, 17645, 25, 185, 1426,
66, 13484, 5991, 11578, 7, 5960, 14025, 4651, 8, 185,
90, 185, 315, 5878, 23740, 7, 87, 49, 1185, 477,
185, 185, 315, 4716, 334, 292, 2140, 12, 29, 2448,
8, 507, 185, 315, 1452, 1439, 62, 13484, 5995, 8110,
25, 185, 436, 967, 1378, 66, 13484, 5995, 8110, 7,
6008, 477, 185, 315, 1452, 1439, 62, 13484, 7256, 4828,
25, 185, 436, 967, 1378, 66, 13484, 7256, 4828, 7,
6008, 477, 185, 315, 1452, 1439, 62, 13484, 17874, 34,
805, 708, 25, 185, 436, 967, 1378, 66, 13484, 17874,
34, 805, 708, 7, 6008, 477, 185, 315, 1452, 1439,
62, 13484, 2826, 15475, 34, 805, 708, 25, 185, 436,
967, 1378, 66, 13484, 2826, 15475, 34, 805, 708, 7,
6008, 477, 185, 315, 1452, 1439, 62, 13484, 3106, 4828,
25, 185, 436, 967, 1378, 66, 13484, 3106, 4828, 7,
6008, 477, 185, 315, 1452, 1439, 62, 13484, 25291, 4828,
25, 185, 436, 967, 1378, 66, 13484, 25291, 4828, 7,
6008, 477, 185, 315, 1452, 1439, 62, 13484, 5991, 508,
4828, 25, 185, 436, 967, 1378, 66, 13484, 5991, 508,
4828, 7, 6008, 477, 185, 315, 1452, 1439, 62, 13484,
13842, 4828, 25, 185, 436, 967, 1378, 66, 13484, 13842,
4828, 7, 6008, 477, 185, 315, 3346, 25, 185, 436,
967, 13147, 4397, 26, 185, 315, 611, 185, 92, 15189,
4535, 1378, 66, 13484, 5991, 11578, 1641, 185, 13518, 21289,
25]], device='cuda:0')
请问造成这种换乱输出的原因可能是什么
from deepseek-coder.
https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/HumanEval/eval_instruct.py 你可以查看这个
这似乎不能解决我的问题,我使用base模型微调的,使用下面的tokenize方式会报错:
inputs = tokenizer.apply_chat_template(
[{'role': 'user', 'content': prompt }],
return_tensors="pt"
).to(model.device)
from deepseek-coder.
https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/HumanEval/eval_instruct.py 你可以查看这个
这似乎不能解决我的问题,我使用base模型微调的,使用下面的tokenize方式会报错: inputs = tokenizer.apply_chat_template( [{'role': 'user', 'content': prompt }], return_tensors="pt" ).to(model.device)
对 这个是chat模型的prompt拼接方式,和coder的不一样
from deepseek-coder.
https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/HumanEval/eval_instruct.py 你可以查看这个
这似乎不能解决我的问题,我使用base模型微调的,使用下面的tokenize方式会报错: inputs = tokenizer.apply_chat_template( [{'role': 'user', 'content': prompt }], return_tensors="pt" ).to(model.device)
对 这个是chat模型的prompt拼接方式,和coder的不一样
我用的base模型,然后依据微调的模版设置的prompt,只让预测Response的内容,然而遇到了输出错乱的问题
from deepseek-coder.
https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/HumanEval/eval_instruct.py 你可以查看这个
这似乎不能解决我的问题,我使用base模型微调的,使用下面的tokenize方式会报错: inputs = tokenizer.apply_chat_template( [{'role': 'user', 'content': prompt }], return_tensors="pt" ).to(model.device)
对 这个是chat模型的prompt拼接方式,和coder的不一样
我用的base模型,然后依据微调的模版设置的prompt,只让预测Response的内容,然而遇到了输出错乱的问题
对 你可以看我提的issue,官方代码预测也是乱输出,我用官方代码微调后的模型,拼接成训练时的prompt也是乱输出
from deepseek-coder.
Related Issues (20)
- 微调完的模型,如何跟基础模型合并? HOT 1
- 本地部署怎么实现vscode自动代码补全? HOT 1
- How to use fine-tuned model? HOT 3
- 使用vllm加速inference后输出容易不符合格式要求
- 使用vllm加载33b-base或33b-instruct后,使用DS-1000、Program-Aided Math Reasoning (PAL)评估集进行评估,得分很低,与论文上的数据不符
- 并发数目
- 用vllm加速推理框架 推理速度还是很慢 HOT 1
- Are EOS tokens masked during pre-training? If so, how does FIM mode know how to connect to the `text_after`?
- 训练数据切分问题 HOT 1
- 疑惑:为什么 base 模型的 tokenizer 词表中也有类似 <|Assistant|> 这样多用于 chat 模型的 special tokens?
- Deepseekcoder 6 spitting out corrupt output for code generation question
- 为什么在进行一次训练加载后,会出现找不到显卡no slot的报错呢?
- RuntimeError: CUDA error: no kernel image is available for execution on the device
- Where is DeepSeek-Coder-V2?
- Long Code Arena
- deepseek-coder-6.7b-base vuejs代码补全上存在一些问题
- 加载模型时出错
- What is the correct padding side for train/eval of base model for FIM?
- dependency parsing code and deduplication script
- Problem in Math Evaluation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepseek-coder.