Comments (5)
ymcui
commented on May 13, 2024
句子数目不一样不影响，粘在一起当作一个长的序列输到BERT里就可以了。
from chinese-bert-wwm.
xudekuan
commented on May 13, 2024
模型是否有长度的限制？如果有限制，规定长度后的都截断了，是不是会影响效果？
from chinese-bert-wwm.
ymcui
commented on May 13, 2024
训练时候的最大长度是512，和原版BERT一直。
至于截断问题你可以参考run_squad.py的做法，将长文本进行切片处理。
https://github.com/google-research/bert/blob/master/run_squad.py
from chinese-bert-wwm.
xudekuan
commented on May 13, 2024
非常感谢您的回复。
from chinese-bert-wwm.
ymcui
commented on May 13, 2024
不客气，本issue关闭。
from chinese-bert-wwm.
Related Issues (20)
- 预训练阶段的loss问题 HOT 2
- 请问一下训练用了几块tpu呀 HOT 2
- 预训练分词是会用到中文vocab后面带“##”的token吗？如果是，整词掩码预训练出来的语言模型，用在下游任务中，下游任务可以不分词吗？ HOT 4
- 有点晕：『开源版本不包含MLM任务的权重』,这句话是什么意思？ HOT 2
- RoBERTa-wwm-ext-large应用到全新领域不收敛 HOT 4
- How to download chinese-roberta-wwm-ext.pt ? HOT 1
- CJRC有预训练的模型么？ HOT 1
- How WWM stratege works in code HOT 2
- 链接失效求助 HOT 2
- RoBERTa-wwm-ext-large ft的时候loss飞了 HOT 2
- 继续预训练 HOT 2
- 如何抽取特定layer的词向量？ HOT 2
- 预训练数据 HOT 2
- 请问可否使用wwm系列的模型做词级别的fill-mask预测？ HOT 2
- NER问题 HOT 2
- Is there any sharing about phoneme-BERT pretrained? HOT 2
- Is onnx model available HOT 2
- 请问每个模型推理时需要多大的显存去加载？ HOT 1
- 请问pytorch版本的chinese-roberta-wwm-ext-large在哪里下载稳定而且快？huggingface上太慢了还总断。 HOT 1
- 从BERT换成此模型后跑不通，应该怎么解决，显示缺少参数 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chinese-bert-wwm.