Comments (5)
I have also come across exploding gradient problem when trying to use mobilenet v2 ssd on Widerface. I am able to bisect the problem to the smooth L1 loss at
pytorch-ssd/vision/nn/multibox_loss.py
Line 45 in 1e86eb3
I suspect it is caused by no default boxes matched. After changing torch.nn.functional.smooth_l1_loss( reduction='sum'), problem is solved.
from pytorch-ssd.
@scarmaten have you tried smaller learning rates?
from pytorch-ssd.
yup i have tried it with learning rate all the way till 1e-20. still no luck.
The problem was partially solved when I removed Expand(mean), RandomSampleCrop(), RandomMirror() under the trainAugmentation in transformation.py but that would mean that the model's accuracy is much worse
from pytorch-ssd.
Hi @scarmaten , one possible way to debug is use a very small, verified dataset as both train and val data to train overfitted models. If you can get a overfitted model with 100% (or almost) accuracy, that means the architecture and system is fine, and the problem might be the data. Otherwise, there is something wrong in the system design.
from pytorch-ssd.
The question then arise, should I adjust the prior boxes for my specific purposes, or just let it go?(As it is kind of working now)
from pytorch-ssd.
Related Issues (20)
- Training doesn't start - I'm getting an error with the data loader HOT 4
- convert_to_caffe2_models.py: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! HOT 2
- SSDSpec for image size smaller than 300 HOT 1
- Re-training SSD-Mobilenet - loss going up and down
- Error using run_ssd_live_demo.py HOT 1
- [Question] what these parameters mean? HOT 2
- base net for mobilenetv3 ssd HOT 1
- ssd-mobilenet to tensorrt
- pytorch模型量化时报错
- Mobilnet-ssdv2 HOT 1
- NAN values in Boxes HOT 1
- How can I train the model without pre-trained weight specified? HOT 1
- runtime error mobillenet-ssd-v2 HOT 5
- run_ssd_example.py box variable type does not mach opencv rectangle and putText functions
- FileNotFoundError: [Errno 2] No such file or directory: 'models/mobilenet_v1_with_relu_69_5.pth'
- cv2.error: OpenCV(4.5.5) : -1 : error: (-5:Bad argument) in function 'rectangle' HOT 1
- Could u add Nesterov momentum in SGD
- 4xgx
- RuntimeError: The size of tensor a (12828) must match the size of tensor b (3000) at non-singleton dimension 1 HOT 3
- ValueError encountered during retraining on Open Images Dataset HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-ssd.