Comments (9)
@zhiqwang really good summary! Thanks for the table. Hopefully we score somewhere in the middle there with our default training script.
Unfortunately the biggest jump they observed was simply from going from 90 epochs to 600 (+ Long Training), but all the small incremental gains do add up from their other experiments.. We don't have many of the more advanced augmentations built in right now, but we should also investigate those. We currently have Weight Decay 1e-5, EMA 0.9999 and Label Smoothing 0.2 enabled by default. Dropout was something I saw in efficientnet_v2, so I put in our Classify() head but have it set to 0.0 by default until we can run more experiments:
Lines 738 to 751 in 0551a31
An additional challenge is that was works really well on ImageNet from scratch is very different than what works well for finetuning smaller datasets on pretrained models, which is probably the most common real-world use case and likely needs separate hyps.
I think this first release will really be about publicising the effort and hopefully getting more eyeballs on classifier.py to allow community contributions to improve the code.
from yolov5.
@zhiqwang yes sort of! We'll have imagenet trained classification models. I'm training them right now, adjusting some settings to get good results. The command I'm using right now is only training imagenet to 30 epochs, but for the final release we'll train to 300 epochs.
The YOLOv5m classifier trains to similar results as ResNet50 and EfficientNet_b0 in 30 epochs, around 0.70 top1 and 0.90 top5, but trains will a bit less CUDA memory and runs a little faster.
!python -m torch.distributed.run --nproc_per_node 4 --master_port 20 classifier.py --pretrained False --data imagenet --epochs 30 --img 224 --batch 512 --device 0,1,2,3 --model yolov5m --lr 0.001 --label-smoothing 0.2
from yolov5.
@parthmalpathak we have an upcoming 6.2 release that should be out in the next few weeks. The major addition there will be classification support migrating to master branch!
from yolov5.
👋 Hello @parthmalpathak, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.
If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.
If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.
For business inquiries or professional support requests please visit https://ultralytics.com or email [email protected].
Requirements
Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:
git clone https://github.com/ultralytics/yolov5 # clone
cd yolov5
pip install -r requirements.txt # install
Environments
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
- Google Colab and Kaggle notebooks with free GPU:
- Google Cloud Deep Learning VM. See GCP Quickstart Guide
- Amazon Deep Learning AMI. See AWS Quickstart Guide
- Docker Image. See Docker Quickstart Guide
Status
If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
from yolov5.
we have an upcoming 6.2 release that should be out in the next few weeks. The major addition there will be classification support migrating to master branch!
Hi @glenn-jocher, Will there be a detection model released pre-trained with the ImageNet datasets when the time comes?
Seems that training from ImageNet can
- Use less epochs and training time
- Use lightweight data augmentation skills, for example, it can be done without the mosaic augmentation.
from yolov5.
The YOLOv5m classifier trains to similar results as ResNet50 and EfficientNet_b0 in 30 epochs, around 0.70 top1 and 0.90 top5, but trains will a bit less CUDA memory and runs a little faster.
It's awesome!
Seems that you're using the following mean and std pairs when training the ImageNet models
Lines 17 to 18 in 0d6490c
If so, it seems that when the detection branch also wants to use this pre-trained model, it would have to use the same mean and std, which would conflict with the (0, 1) pairs used by the current mechanism?
from yolov5.
Just FYI @glenn-jocher , TorchVision team has previously summarized an article for training SOTA models on ImageNet https://pytorch.org/blog/how-to-train-state-of-the-art-models-using-torchvision-latest-primitives/
Accuracy@1 | Accuracy@5 | Incremental Diff | Absolute Diff | |
---|---|---|---|---|
ResNet50 Baseline | 76.130 | 92.862 | 0.000 | 0.000 |
+ LR optimizations | 76.494 | 93.198 | 0.364 | 0.364 |
+ TrivialAugment | 76.806 | 93.272 | 0.312 | 0.676 |
+ Long Training | 78.606 | 94.052 | 1.800 | 2.476 |
+ Random Erasing | 78.796 | 94.094 | 0.190 | 2.666 |
+ Label Smoothing | 79.114 | 94.374 | 0.318 | 2.984 |
+ Mixup | 79.232 | 94.536 | 0.118 | 3.102 |
+ Cutmix | 79.510 | 94.642 | 0.278 | 3.380 |
+ Weight Decay tuning | 80.036 | 94.746 | 0.526 | 3.906 |
+ FixRes mitigations | 80.196 | 94.672 | 0.160 | 4.066 |
+ EMA | 80.450 | 94.908 | 0.254 | 4.320 |
+ Inference Resize tuning * | 80.674 | 95.166 | 0.224 | 4.544 |
+ Repeated Augmentation ** | 80.858 | 95.434 | 0.184 | 4.728 |
* The tuning of the inference size was done on top of the last model.
** Community contribution done after the release of the article.
from yolov5.
I think this first release will really be about publicising the effort and hopefully getting more eyeballs on classifier.py to allow community contributions to improve the code.
Yep, that's a great start! And looking forward to more models being integrated into yolov5.
from yolov5.
@zhiqwang absolutely! We're excited about the journey ahead and appreciate your support. Keep an eye out for more improvements and additions in the future. Thank you for your valuable input!
from yolov5.
Related Issues (20)
- How can I save the detections Yolov5 makes when he's working with a camera source? HOT 4
- How to specify yolov5 to train multiple folders? HOT 1
- pulling out model's layer intermediates HOT 2
- Continuous training of a Ultralytics Model HOT 4
- Exporting trained yolov5 model (trained on custom dataset) to 'saved model' format changes the no. of classes and the name of classes to default coco128 values HOT 2
- more details about training procedure HOT 4
- divide the objects into small and large categories based on the size of the bonding boxes HOT 8
- Request for YOLOv5 v6.2 Source Code under GPL-3.0 License HOT 4
- What prevents me from using the AMP function? HOT 4
- What prevents me from using the AMP function? HOT 1
- What prevents me from using the AMP function? HOT 1
- Background annotation HOT 6
- Hi @7rkMnpl, HOT 2
- Multiple GPU Hyperparameter evolution HOT 5
- Marking YOLOv5 Detection Text Outputs with TP or FP HOT 4
- Multiple threads using yolov5 model concurrent inference failed HOT 4
- Detect head structure differs HOT 4
- runs\train\exp10 is not a directory HOT 12
- Similar mAP when splitting data into train, val and test HOT 4
- Syntax and understanding questions about reading tensorflow lite results HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolov5.