Comments (3)
DLLXW
commented on July 20, 2024
很多评测样例其实出现在了SFT数据中，所以让我误以为模型具备很流畅的问答能力
这个对于生产其实问题不大，这说明对于生产所需的问答对，也能流畅问答了。我是没想到 50M 就能用了，而平时用 7B 的都笨得要死。
离生产可用差太远了。我用的是一个比赛数据集，由于一些协议原因我暂时没法把数据集开源出来哈。比赛地址：https://competition.huaweicloud.com/information/1000041928/html12。比赛数据3000条左右吧，我留了100条验证，这个比赛的blue的话，在～0.02x，初赛排行榜上50名开外吧哈哈哈，主要是参数量太小了。人肉评测的话。基本可以听懂人类指令和意图，但是回答的都是东拼西凑的，正确性很低。
from baby-llama2-chinese.
linonetwo
commented on July 20, 2024
好吧，所以说至少还是得上 b 了，M 级的难说。我有空自己试试…
from baby-llama2-chinese.
franklyd
commented on July 20, 2024
多大的参数规模会好一些呢？
from baby-llama2-chinese.
Related Issues (20)
- Ignore the `freqs_cis` buffer so that DDP does not broadcast it at construction time
- 为了丰富和扩充本项目，这里开源了使用deepspeed进行训练的代码和权重（1.75B）
- 请问下这个报错是什么信息？
- 请问下这个报错是哪里配置的不对吗？
- Problem with tokenizer? HOT 3
- 请问单卡16G显存的4060Ti能训练吗？ HOT 1
- 关于运行一段时间，机器断电，如何继续训练 HOT 2
- c4-zh数据有问题 HOT 3
- 预训练模型参数和eval参数维度不匹配的问题
- 交个作业吧
- 请问支持tensorrt llm部署吗
- 作者，这个项目支持断点续训嘛 HOT 2
- 请问在处理微调数据集时为何要限制文本长度? HOT 1
- 预训练阶段，每条训练样本混杂着不同的句子（不同句子用<eos>隔开）
- chatglm_tokenizer 模块是在哪个软件包中？ HOT 2
- 请问哪步加的 Positional embeddings HOT 1
- 请问大数据量怎么加载呢？
- 请问语言模型的强化学习有可以参考的开源项目吗？
- smallvocab tokenizer
- 模型的回答较长，输出结果不完整要怎么解决
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from baby-llama2-chinese.