Comments (12)
Thanks for the high quality references! I can confirm that your default version of PWC-Net corresponds to PWC-Net ROB of the paper "Model Matters..." (https://arxiv.org/pdf/1809.05571.pdf)
Trained on Sintel and scoring exactly 1.81 px EPE on sintel training clean.
Congratulations for your great work!
Stefano
from pytorch-pwc.
Thanks for sharing your findings! So it seems like you got better results than the paper claimed as long as you use a recent PyTorch version? I wonder why this is the case, should you find any issues in my implementation then please don't hesitate to let me know.
from pytorch-pwc.
Oh wow, nice find and thanks for sharing this information!
from pytorch-pwc.
Thank you for your feedback! What is the EPE when you use the official repository? Specifically, the pwc_net.pth.tar
and the pwc_net_chairs.pth.tar
models. Thanks!
from pytorch-pwc.
Unfortunately I cannot run the official network. It requires CUDA 8.0, but in my cluster I am forced to use CUDA 10. (if you have any hint on how to make it work it would be greatly appreciated)
So I run your net on the 1041 samples of "Sintel training clean" and I got 1.81 EPE.
According to the paper, the closest EPE value to the one I got is (red circle): meaning this could be the version fine-tuned on sintel.
Could you help me clarifying which version we are using?
NOTE: my result 1.81 actually seems to match the value reported in your link, so I think your network is working properly: https://github.com/NVlabs/PWC-Net/tree/master/PyTorch
But in the link it is not specified which version it is.
Thanks for your help,
Stefano
from pytorch-pwc.
This comment might be useful: NVlabs/PWC-Net#26 (comment)
However, I do not think that there is a corresponding entry in the paper. But please correct me if I am wrong, I am sure others would be happy to get an answer to this question as well!
For a related discussion, please see: #9
from pytorch-pwc.
Awesome, thank you for the clarification!
from pytorch-pwc.
I manged to install Cudatools 9.2 but my card was too new for Cudatools 8.0 plus I was having to change my code too much for the older versions of PyTorch.
I still couldn't reproduce the same numbers as in the paper but at least I seem to be in the ballpark. It could be because I have Cuda 11.6.1 installed. I don't feel like downgrading that installation too and there is no longer a MiniConda for Python 2.7 available.
Older versions of nn.functional.grid_sample
don't support the align_corners
attribute but supposedly it was true before 1.3.0 and it is set to false in the latest version of the code. When I tried PyTorch 1.4.0 with Cudatools 9.2/10.0 I got the same results as PyTorch 1.10.2.
Cudatools / PyTorch / Conda | Simtel Clean EPE (training) |
Simtel Final EPE (training) |
---|---|---|
8.0 / 0.2 / 2.7 | (1.81) | (2.29) |
9.2 / 1.1.0 / 37_4.11 | 2.80980 | 3.28290 |
10.2 / 1.10.2 / 39_4.11 | 1.49756 | 1.94308 |
Note: Numbers in parenthesis are from the paper for PWC-Net_ROB
from pytorch-pwc.
Thanks for sharing your findings! So it seems like you got better results than the paper claimed as long as you use a recent PyTorch version? I wonder why this is the case, should you find any issues in my implementation then please don't hesitate to let me know.
The difference in PyTorch versions seemed to be the align_corners
parameter setting. The not matching the paper was probably my Nvidia CUDA Toolkit version which I didn't change for these tests. I'm sure Nvidia made improvements in the past few years. Perhaps if I had used CUDA Toolkit 9.2 along with cudatools=9.2
and PyTorch 1.4.0. I could've matched the paper.
I just do this as a hobby. Not like any of my results will be published in a paper but I will put it out here on Github. My goal is to have a lot better performance than the paper by changing up the learning even more. I've learned a lot in the past year though and I still find this stuff interesting. But if I find any more issues I'll be happy to share.
from pytorch-pwc.
You can try the following backwarp
when using an older PyTorch that doesn't support align_corners=False
.
def backwarp(tenInput, tenFlow):
if str(tenFlow.shape) not in backwarp_tenGrid:
tenHor = torch.linspace(-1.0, 1.0, tenFlow.shape[3]).view(1, 1, 1, -1).repeat(1, 1, tenFlow.shape[2], 1)
tenVer = torch.linspace(-1.0, 1.0, tenFlow.shape[2]).view(1, 1, -1, 1).repeat(1, 1, 1, tenFlow.shape[3])
backwarp_tenGrid[str(tenFlow.shape)] = torch.cat([ tenHor, tenVer ], 1).cuda()
# end
if str(tenFlow.shape) not in backwarp_tenPartial:
backwarp_tenPartial[str(tenFlow.shape)] = tenFlow.new_ones([ tenFlow.shape[0], 1, tenFlow.shape[2], tenFlow.shape[3] ])
# end
tenFlow = torch.cat([ tenFlow[:, 0:1, :, :] / ((tenInput.shape[3] - 1.0) / 2.0), tenFlow[:, 1:2, :, :] / ((tenInput.shape[2] - 1.0) / 2.0) ], 1)
tenInput = torch.cat([ tenInput, backwarp_tenPartial[str(tenFlow.shape)] ], 1)
tenOutput = torch.nn.functional.grid_sample(input=tenInput, grid=(backwarp_tenGrid[str(tenFlow.shape)] + tenFlow).permute(0, 2, 3, 1), mode='bilinear', padding_mode='zeros')
tenMask = tenOutput[:, -1:, :, :]; tenMask[tenMask > 0.999] = 1.0; tenMask[tenMask < 1.0] = 0.0
return tenOutput[:, :-1, :, :] * tenMask
# end
from pytorch-pwc.
Cool. You have my curiosity so I'll try that
from pytorch-pwc.
I was wrong about the Nvidia Cuda Toolkit... It doesn't even need to be installed unless you want to compile the code. Looks like PyTorch provides the runtime libraries it needs.
It seems they center cropped the images for training and didn't bother to test them without cropping.
Cudatools / PyTorch / Conda | Simtel Clean EPE (training) |
Simtel Final EPE (training) |
---|---|---|
8.0 / 0.2 / 2.7 | (1.81)1 | (2.29) |
9.0 / 1.1.0 / 37_4.11 | 1.47732 | 1.93140 |
9.0 / 1.1.0 / 37_4.11 | 1.843212 | 2.277692 |
10.2 / 1.10.2 / 39_4.11 | 1.49756 | 1.94308 |
10.2 / 1.10.2 / 39_4.11 | 1.866612 | 2.294352 |
10.2 / 1.10.2 / 39_4.113 | 1.843212 | 2.277692 |
Notes:
1. Numbers in parenthesis are from their paper for PWC-Net_ROB (Table 1)
2. Cropped to 768x320 per their paper (Section 4.1.3)
3. align_corners=True
and same linspace
as that was used for PyTorch 1.1.0
I used the following to toggle between the two versions of linspace
and grid_sample
from distutils.version import LooseVersion
if LooseVersion(torch.__version__) >= LooseVersion('1.3'):
from pytorch-pwc.
Related Issues (20)
- Can it test in CPU device? HOT 1
- Cannot inspect the model using TensorBoard HOT 4
- Generalization to unseen data HOT 1
- how to compute optical flow for small size imgs? HOT 1
- what is the range of optical flow value in an image? HOT 1
- About the direction of estimated flow HOT 2
- About pwc-net pretrained model in pytorch version HOT 1
- Just want to confirm if my analysis of your commit 5f4d7de is correct HOT 7
- In run.py, self.netSix has out_channels = 196. Paper says 192. HOT 1
- Receiving a random error, CUDA_ERROR_ILLEGAL_ADDRESS HOT 9
- CuPy correlation layer error HOT 4
- Cupy cuda error. HOT 1
- What is the purpose of `tenMask` in `run.backwarp`? HOT 2
- the two same images HOT 4
- I already did what @fabiopk said but I am still getting this error HOT 2
- How to change the cuda version of correlation.py to the python version HOT 1
- how to import frames of river videos into PWC-Net codes HOT 7
- cupy issue HOT 1
- Normalization in `backward` function. HOT 1
- Deprecation of cupy.cuda.compile_with_cache() in cupy 13.0 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-pwc.