Comments (7)
首先感谢开源 Qwen-7B 模型,我基于该模型实现了 QLoRA 多轮对话微调,项目地址:https://github.com/hiyouga/LLaMA-Efficient-Tuning
QLoRA 指令微调:
CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \ --stage sft \ --model_name_or_path Qwen/Qwen-7B-Chat \ --do_train \ --dataset sharegpt_zh \ --template chatml \ --finetuning_type lora \ --lora_target c_attn \ --output_dir qwen_lora \ --per_device_train_batch_size 4 \ --gradient_accumulation_steps 4 \ --lr_scheduler_type cosine \ --logging_steps 10 \ --save_steps 100 \ --learning_rate 3e-5 \ --num_train_epochs 1.0 \ --quantization_bit 4 \ --fp16Web Demo:
python src/web_demo.py \ --model_name_or_path Qwen/Qwen-7B-Chat \ --template chatmlAPI 部署(基于 OpenAI 格式):
python src/api_demo.py \ --model_name_or_path Qwen/Qwen-7B-Chat \ --template chatml
另外,希望开发者可以修复一下 tokenizer 的 decode 方法,使其支持 skip_special_tokens 参数,便于后续开发,目前该参数没有实际生效。(最新版已修复)def _decode( self, token_ids: Union[int, List[int]], skip_special_tokens: bool = False, clean_up_tokenization_spaces: bool = None, **kwargs, ) -> str: if isinstance(token_ids, int): token_ids = [token_ids] return self.tokenizer.decode(token_ids)
ValueError: Encountered text corresponding to disallowed special token '<|im_start|>'.
If you want this text to be encoded as a special token, pass it to allowed_special
, e.g. allowed_special={'<|im_start|>', ...}
.
If you want this text to be encoded as normal text, disable the check for this token by passing disallowed_special=(enc.special_tokens_set - {'<|im_start|>'})
.
To disable this check for all special tokens, pass disallowed_special=()
.
from qwen.
首先感谢开源 Qwen-7B 模型,我基于该模型实现了 QLoRA 多轮对话微调,项目地址:https://github.com/hiyouga/LLaMA-Efficient-Tuning
QLoRA 指令微调:CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \ --stage sft \ --model_name_or_path Qwen/Qwen-7B-Chat \ --do_train \ --dataset sharegpt_zh \ --template chatml \ --finetuning_type lora \ --lora_target c_attn \ --output_dir qwen_lora \ --per_device_train_batch_size 4 \ --gradient_accumulation_steps 4 \ --lr_scheduler_type cosine \ --logging_steps 10 \ --save_steps 100 \ --learning_rate 3e-5 \ --num_train_epochs 1.0 \ --quantization_bit 4 \ --fp16Web Demo:
python src/web_demo.py \ --model_name_or_path Qwen/Qwen-7B-Chat \ --template chatmlAPI 部署(基于 OpenAI 格式):
python src/api_demo.py \ --model_name_or_path Qwen/Qwen-7B-Chat \ --template chatml
另外,希望开发者可以修复一下 tokenizer 的 decode 方法,使其支持 skip_special_tokens 参数,便于后续开发,目前该参数没有实际生效。(最新版已修复)
源码对应位置:huggingface.co/Qwen/Qwen-7B-Chat/blob/5e7f6a3f41724e7cb8ea3e3be7a1faf2bd5d6a38/tokenization_qwen.py#L228def _decode( self, token_ids: Union[int, List[int]], skip_special_tokens: bool = False, clean_up_tokenization_spaces: bool = None, **kwargs, ) -> str: if isinstance(token_ids, int): token_ids = [token_ids] return self.tokenizer.decode(token_ids)ValueError: Encountered text corresponding to disallowed special token '<|im_start|>'. If you want this text to be encoded as a special token, pass it to
allowed_special
, e.g.allowed_special={'<|im_start|>', ...}
. If you want this text to be encoded as normal text, disable the check for this token by passingdisallowed_special=(enc.special_tokens_set - {'<|im_start|>'})
. To disable this check for all special tokens, passdisallowed_special=()
.
我能成功运行,加载了Qwen7B .但是如果是openAI格式的API, 客户端的api key填什么呢?
from qwen.
@stuarthe 留空
from qwen.
@stuarthe 留空
嗯,已成功连接。谢谢!
from qwen.
@stuarthe 留空
求通过 llama efficient tuning 的PR, 解决了 bos token的问题
from qwen.
mark
from qwen.
Lora微调后的Qwen模型根本不能直接调用chat接口!报错 generation_config缺少chat_ml字段
from qwen.
Related Issues (20)
- [BUG] <title>fastchat + vLLM +OpenAI API 调用qwen模型,数据不需要预先处理吗 HOT 1
- 本地部署后,运行很慢啊 HOT 4
- 请问下 2.5什么时候开源呀? HOT 1
- File "finetune.py", line 412, in <module> train() File "finetune.py", line 384, in train model = get_peft_model(model, lora_config) File "/opt/conda/envs/qwen/lib/python3.8/site-packages/peft/mapping.py", line 123, in get_peft_model peft_config.base_model_name_or_path = model.__dict__.get("name_or_path", None) AttributeError: 'NoneType' object has no attribute '__dict__'[BUG] <title> HOT 2
- qwen 14b 不微调的情况下,问相同的问题,模型输出也不太一致,是为什么?温度已经设置成0了 HOT 2
- [BUG] <title>torch.cuda.OutOfMemoryError: CUDA out of memory. HOT 1
- Qwen pre_trained, 打印一下内容,就没有了,不确定是否训练完成 HOT 2
- [BUG] 转换Qwen1.5-14B报错 HOT 1
- 多轮对话训练数据格式组织 HOT 1
- [BUG] Questionable embedding feature shape extracted from Qwen-7B-Chat HOT 2
- [BUG] <title> 命令行运行参数解析错误
- 工具调用的时候,本来用户没有输入参数,但是模型会自动幻想参数 HOT 2
- [BUG] model的forward函数接收attention_mask的时候,若attention_mask[i, 0]==0,则序列i输出的logits全都是NaN值 HOT 6
- 模型的TEMPLATE是怎么样的 HOT 1
- [BUG] <title>全参数微调qwen-14b-chat时卡住 HOT 1
- 运行web_demo.py程序时问答卡顿 HOT 1
- [BUG] <title>Qwen有支持OpenAI形式的functoncall的计划吗 HOT 1
- [BUG] <title>RuntimeError: Expected attn_mask dtype to be bool or to match query dtype, but got attn_mask.dtype: c10::BFloat16 and query.dtype: c10::Half instead. HOT 1
- 量化细节请教 HOT 1
- [BUG] 这模型似乎很固执或直男癌,prompt里明确了不要怎么怎么样,每次输出还是不按要求去 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qwen.