ignotus / theano-flownet

FlowNetS and FlowNetC port to Theano

theano-flownet

This is a port of the caffe implementation of the ICCV'15 paper "FlowNet: Learning Optical Flow with Convolutional Networks" by Dosovitskiy et al to Theano and Lasagne. It contains both FlowNetS and FlowNetC models and a port of the correlation layer.

Ground Truth	FlowNetS	FlowNetC

Requirements

• flownet-caffe (for the weight conversion script only).
• Theano 0.8.2
• Lasagne
• numpy
• flow-io-opencv can be used to generate optical flow visualization from *.flo outputs

Setup

caffe_to_numpy.py script can be used to convert caffe models to the npz format. caffemodel and prototxt files should be placed in the model subdirectory. Alternatively you can download weights from Google Drive.

Running

python FlowNetS.py
python FlowNetC.py

License

The source code is distributed under the MIT license. Please refer to us if you find this code useful.

theano-flownet's Issues

Batch Norm

In the original flownet implementation there is batch normalization after every conv layer. In theFloatNetC.py, I don't find any batch_norm layer. Any specific reason?

flip_filters = True / False

When training the theano-flownet, I observed that using the original code causes exploding gradients. However, after setting flip_filters option in leaky_deconv() and upsample() layer to False, the Theano training works.

So I just wonder is the flip_filters=true necessary for leaky_deconv() and upsample() ? Is this option for importing Caffe weights?

Thank you

Could you please upload the flownetc.npz file to somewhere like Google Drive?

I am transforming the model to tensorflow platform. These .npz file will help a lot. Thank you!