Comments (5)
Yeah, thanks for the question, for the large dataset, it will be overfitting very quickly, so you can set the epoch to a small value, actually, 20 to 100 is a good choice, and you can also apply the early stop with patience 7 to 20 ~
from multibench.
Thanks for your reply!
When i train my own model, I found that after 100 epoch training , the valid loss from 0.76 changed to 0.62. It seems that after training, the model effect is not significantly improved, is this normal? @Vanvan2017
from multibench.
One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.
from multibench.
So should I use some classification loss functions to train my model?
By the way, why you train the classification task as a regression task? Is this a common practice?
Thanks!
One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.
from multibench.
So should I use some classification loss functions to train my model? By the way, why you train the classification task as a regression task? Is this a common practice?
Thanks!
One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.
Sure, you can use BCE or CrossEntropy to train that, but by default, the MOSEI is a regression dataset, and previous literature uses regression loss (MAE), it may loss the information of how strong the sentiments when just using 0 or 1 label I think. But actually, take everything into traditional classification setting is also a good standard way to evaluate your own model.
from multibench.
Related Issues (20)
- It will be great to have some tests to make sure this benchmark is runnable. HOT 27
- questions about mosei dataset HOT 5
- [QUESTION] Availability of trained models HOT 1
- What's the meaning of modalities in MUJOCO PUSH dataset? HOT 2
- Questions about the video encodings of the mosi and mosei datasets HOT 1
- Code to obtain features from raw data HOT 1
- Leaderboard
- Did you forget switch model train/eval state? HOT 1
- Errors running mmimdb examples HOT 2
- Models, data used in get_data.py for mmimdb missing
- Labels for CMU MOSEI HOT 1
- can you make multibench a python package?
- Question about relative robustness
- Extracting info from the H5 files HOT 6
- Suggestion about PyTorch version HOT 1
- Problem with code: os.environ['CUDA_VISIBLE_DEVICES'] = '0'
- The link to the raw dataset is broken HOT 4
- Info regarding preprocessed MOSEI
- Questions about the im.pk data & MFM HOT 2
- Question about the mortality label on im.pk file HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multibench.