aranganath / exploratory Goto Github PK

Now that we have created the flow for two corresponding images in the last issue, let's start working on creating images in consecutive images.

Let's say we created the flow from images 0000.png and 0001.png.

We now need to create the flow from 0001.png and 0002.png.

Doing so, we need to create the flow between subsequent flows from x.png and x+1.png (where x is of the form XXXX where X is between [0,9]).

To do this, first:

1. Save the .flo file for each subsequent image in a 'flofolder'.
2. Use the flowiz api (here) to create the flow image.
3. Save this flo image into another folder named 'floimages'

Tasks for the next week

• Install conda for environment management:
• Install all packages (and dependencies): pytorch, numpy, scipy, opencv etc.: (https://pytorch.org/get-started/locally/, https://numpy.org/install/, https://scipy.org/install/, https://pypi.org/project/opencv-python/)
• Run the pytorch code for training neural networks on mnist (You may use the code given here: https://nextjournal.com/gkoehler/pytorch-mnist)

The purpose is to understand how neural networks work.

Deadline: Monday (May 23rd, 2022)

The following are the goals for next week:

1. Install conda, Pytorch and all related packages. (May use any OS, but personal help will be provided if it is linux)
2. Design a neural network with 3 convolutional layers and 2 fully connected layers.
3. Train the network on the MNIST dataset.

Now that preliminaries are out of the way, the next step is to extract the flow from a video.

1. Understand what is happening in the paper: https://arxiv.org/pdf/1709.02371.pdf
2. Either implement it from scratch, or find a network that does it (pre-designed network)
3. Train the model on some video files and see what the results look like

Estimated time of completion: 2 weeks (5 days extra)

You can find the implementation here: https://github.com/NVlabs/PWC-Net

For the Pytorch implementation, please look at: https://github.com/sniklaus/pytorch-pwc

Also install flowiz from here : https://github.com/georgegach/flowiz (pip install flowiz -U)

After a week:

You should be able to implement and generate the flow.

Now, download the GOPRO dataset from here: https://seungjunnah.github.io/Datasets/gopro (either GoPro or GoPro_Large)

Explore what to do with it.