Comments (5)
airaria
commented on May 27, 2024
什么任务，什么配置？
from textbrewer.
hahlw
commented on May 27, 2024
什么任务，什么配置？
文本分类任务，单卡TITAN XP(12g) 训练了30个epoch,选用比较好的epoch测试是75.94%，bertbase是80.02%（之前的指标观测错了）这个指标算是正常吗？我加了attentionloss 但是loss是先上升后下降。这个正常吗？
from textbrewer.
airaria
commented on May 27, 2024
看起来正常，如果没调过超参，再调一调应该还有上升空间
from textbrewer.
stale
commented on May 27, 2024
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from textbrewer.
stale
commented on May 27, 2024
Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.
from textbrewer.
Related Issues (20)
- pre-trained student weights HOT 3
- Where to find gs4210.pkl file or how to generate it ? thanks HOT 2
- interpreting intermediate matches HOT 5
- Show the progress bar when training. HOT 3
- Picking right layers HOT 3
- How about the distillation effect of gpt2 ？ HOT 2
- Does it support translation model? HOT 2
- 在VisionTransformer HOT 7
- 关于ner数据的处理 HOT 2
- notebook_examples/msra_ner.ipynb 运行报错 HOT 12
- 不同维度蒸馏有对应的例子吗，从768降到256 HOT 4
- msra_ner.ipynb最后的trainer.evaluate()显示CUDA out of memory，请问训练的显存要求是多大？十分感谢！ HOT 2
- 老师，您好，请问有多任务多教师的蒸馏的demo吗？ HOT 4
- 老师您好，我想问一下，比如roberta蒸馏到tinybert，中间的hidden是通过线性层拉到同样的维度去算mse，那在推理的时候岂不是这些经过梯度更新的线性层毫无作用？那请问这些线性层仅仅就是为了调整维度？ HOT 2
- 蒸馏后的模型进行evaluate，报错AxisError: axis 2 is out of bounds for array of dimension 1 HOT 5
- 可以使用chatgpt蒸馏到bert或者T5吗？ HOT 2
- 麻烦问下，目前支持llama模型吗 HOT 2
- 请问支持BERT-of-Theseus的蒸馏方式吗 HOT 3
- TextBrewer/examples/notebook_examples/msra_ner.ipynb have bug? HOT 1
- TextBrewer/src/textbrewer/distiller_utils.py get_outputs_from_batch fails tocheck dicts properly for maskedLM HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from textbrewer.