Coder Social home page Coder Social logo

njuvision / awnet Goto Github PK

View Code? Open in Web Editor NEW
17.0 17.0 6.0 6.5 MB

A Dual Camera System for High Spatiotemporal Resolution Video Acquisition (TPAMI 2020)

Home Page: https://NJUVISION.github.io/AWnet

License: MIT License

Python 43.33% Shell 0.37% C++ 9.22% Cuda 29.64% HTML 17.44%

awnet's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

awnet's Issues

Non differentiable Warping

Hi
I am playing around with your code and I learnt that these lines are non-differentiable and do not let the gradient compute. Can you please guide me on how you solved that? or its just me who is having this issue.

    mask = torch.autograd.Variable(torch.ones(x.size())).cuda()
    mask = nn.functional.grid_sample(mask, vgrid)

   # if W==128:
        # np.save('mask.npy', mask.cpu().data.numpy())
        # np.save('warp.npy', output.cpu().data.numpy())

    mask[mask<0.9999] = 0
    mask[mask>0] = 1

    return output*mask,mask

for the simplest code to reproduce the error,

import torch
use_cuda = torch.cuda.is_available()
device = torch.device("cuda" if use_cuda else "cpu")
torch.cuda.set_device(0)
criteria = torch.nn.L1Loss()


# import awnet_pwcnet #please adjust as per your code

net = awnet_pwcnet.PWCDCNet().cuda()
#net = torch.nn.Conv2d(6, 3, 1).cuda()
optimizer = torch.optim.Adam(net.parameters(), lr = 0.1)

inp = Variable(torch.rand(1,3,256,448).to(device),requires_grad = True)
snd = Variable(torch.rand(1,3,256,448).to(device),requires_grad = True)
gt = Variable(torch.rand(1,2,256,448).to(device),requires_grad = True)

x = torch.cat((inp,snd),1)

# for layer in net.children():
#     for param in layer.parameters():
#         print(param)

for i in range(1,5):
    optimizer.zero_grad()
    out = net(x)
    out = F.interpolate(out,scale_factor=4,mode='bilinear',align_corners=False)*20
    loss = criteria(out,gt)
    print(out.shape)
    loss.backward()#retain_graph= True)
    print(net.state_dict())
    optimizer.step()

and i get this error which is caused by these lines as when I replaced the warping function with this one, it started to train.
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Thanks alot.

Evaluation toolkit

Hi,

Can you please guide me which toolkit you used for the calculation of evaluation metrics? SSIM/PSNR?

Thanks a lot.

Pre-trained Weights

Hi,

Its an amazing work. please upload the pre-trained weights to play around with. Thanks.

About the pretrained pwc-net

Hello, in the paper you mentioned using pretrained PWC_Net to initialize our FlowNet, I want to ask how to get this pretrained PWC_Net? Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.