Comments (8)
Yes, it may seem strange.
But, at 500 epochs, all the models are not converged, I have to choose one value to compare.
So I chose the best one of each model.
I run all the other models in the same way, from this perspective the comparison is fair.
And when you run enough epochs (over 500), the current valid PSNR will be very close to best one.
Sorry to confuse you.
from estrnn.
I have added the test code for the result generation.
Thank you.
from estrnn.
I have added the test code for the result generation.
Thank you.
Thank you for uploading the test code.
However, I have still some issues with training and test codes.
Please comment on these issues.
-
Test accuracy is much lower than valid accuracy during training. Visual results are also not sharp.
For a single GPU model, the test PSNR is 29.83dB while the best validation PSNR is 30.73dB.
I trained the model with the old source code. -
Model performance (validation accuracies) seems to depend on the number of GPUs used in training. (used ddp mode)
-
Example logs)
-
1 GPU used
2020/09/17, 15:37:23 - [Epoch 500 / lr 2.50e-05]
[train] epoch time: 401.73s, average batch time: 0.84s
[train] 1MSE : 65.95 (best 62.06), PSNR : 31.84 (best 32.14)
[valid] epoch time: 93.00s, average batch time: 0.37s
[valid] 1MSE : 89.95 (best 80.37), PSNR : 30.22 (best 30.73) -
2 GPUs used
2020/09/19, 18:02:29 - [Epoch 500 / lr 2.50e-05]
[train] epoch time: 179.98s, average batch time: 0.75s
[train] 1MSE : 71.04 (best 68.76), PSNR : 31.23 (best 31.40)
[valid] epoch time: 36.07s, average batch time: 0.28s
[valid] 1MSE : 95.21 (best 88.25), PSNR : 29.76 (best 30.21) -
4 GPUs used
2020/09/18, 05:00:53 - [Epoch 500 / lr 2.50e-05]
[train] epoch time: 103.80s, average batch time: 0.86s
[train] 1MSE : 95.22 (best 85.12), PSNR : 30.19 (best 30.54)
[valid] epoch time: 22.33s, average batch time: 0.35s
[valid] 1MSE : 110.71 (best 99.94), PSNR : 29.06 (best 29.52)
from estrnn.
As for issue #1:
The PSNR values for our model and other models are all calculated through one round of random cropped (256x256) test dataloader (Not on full resolution video). What we want to compare is the deblurring efficiency of each model under same experimental conditions. Actually, if you want to get sharper results, you could increase the training epochs since the model is not converged at 500 epochs. And you can use larger model configuration, e.g., increasing "n_blocks" and "n_feats" as we did in the paper.
As for issue #2:
If you use ddp mode, the real batch size is "num_gpus*batch_size", which means each process (GPU) will use the batch size you set in "para". Batch size will affect the performance of the model. In general, small batch size is better with the same training epochs, but I believe the effect of hyper-paramter "frames" is coupled to batch size. This also applies to other model like IFIRNN.
Thank you.
from estrnn.
Thank you for the fast reply
For issue #2, I see your reply.
However, for issue #1, my training setting is the same as the model (B9C80) in the paper, which is trained for 500 epoch and shows the PSNR 30.79 dB.
Validation accuracy is similar to the paper's value although its accuracy is calculated from random crops.
My test accuracy is much lower than the paper's value.
I think that there may be a problem with the test code.
Could you provide the pretrained model or check again that the current test code is valid?
from estrnn.
I didn't write the code in the test function to calculate PSNR since the validation and test in GOPRO is same.
Could you please send me your code to calculate PSNR and your (B9C80) checkpoint?
My email: [email protected]
Thank you
from estrnn.
You mean that PSNR values in Table. 1 in the paper are validation accuracies, calculated from random crops?
Thank you
from estrnn.
OK I see, my question is settled.
Thank you.
I will close this issue.
from estrnn.
Related Issues (20)
- about dataset HOT 3
- About pretrained model HOT 3
- About training time of BSD dataset HOT 2
- How to use with custom input ?? HOT 2
- NotImplementedError: There were no tensor arguments to this function HOT 6
- Are these artifacts expected? HOT 4
- Training problems HOT 2
- About training HOT 1
- Cannot download BSD dataset HOT 2
- Using pre-trained model to inference , the results is strange HOT 6
- About dataset HOT 1
- Difference of GMACs HOT 1
- Cannot download BSD HOT 8
- Issue on downloading BSD database HOT 4
- CPU HOT 2
- About Dataset HOT 1
- When i want to train in my own dataset to deblur, the value of loss run into nan HOT 1
- About test datasets settings HOT 1
- Is it possible to provide the full BSD with a RAW Baidu Cloud download link?
- Test Output to video
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from estrnn.