qijiezhao / py-denseflow

Extract TVL1 optical flows in python (multi-process && multi-server)

Py-denseflow

This is a python port of denseflow, which extract the videos' frames and optical flow images with TVL1 algorithm as default.

---

Requirements:

• numpy
• cv2
• PIL.Image
• multiprocess
• scikit-video (optional)
• scipy

Installation

Install the requirements:

pip install -r requirements.txt

---

Usage

The denseflow.py contains two modes including 'run' and 'debug'.

here 'debug' is built for debugging the video paths and video-read methods. (IPython.embed suggested)

Just simply run the following code:

python denseflow.py --new_dir=denseflow_py --num_workers=1 --step=1 --bound=20 --mode=debug

While in 'run' mode, here we provide multi-process as well as multi-server with manually s_/e_ IDs setting.

for example: server 0 need to process 3000 videos with 4 processes parallelly working:

python denseflow.py --new_dir=denseflow_py --num_workers=4 --step=1 --bound=20 --mode=run --s_=0 --e_=3000

---

Just feel free to let me know if any bugs exist.

py-denseflow's Issues

Error in debug mode

video read methods are not working. I have the following error
AttributeError: module 'cv2.cv2' has no attribute 'createOptFlow_DualTVL1'

How to fix this?

How can i use GPU through your code

Thanks for your contribution.

I have a question about the GPU use. How can I use GPU through your code?

I reckon original dense_flow can use GPU to accelerate the speed.

NameError: name 'videos_root' is not defined

Hello! very useful code. However when I run the code I get the above error. Do you know why that is and can I resolve it?

`runfile('D:/Clones/py-denseflow-master/denseflow1.py', wdir='D:/Clones/py-denseflow-master')
find 13320 videos.
get videos list done!
Traceback (most recent call last):

File "", line 1, in
runfile('D:/Clones/py-denseflow-master/denseflow1.py', wdir='D:/Clones/py-denseflow-master')

File "C:\Users\sancy\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)

File "C:\Users\sancy\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/Clones/py-denseflow-master/denseflow1.py", line 184, in
pool.map(dense_flow,zip(video_list,flows_dirs,[step]*len(video_list),[bound]*len(video_list)))

File "C:\Users\sancy\Anaconda3\lib\multiprocessing\pool.py", line 290, in map
return self._map_async(func, iterable, mapstar, chunksize).get()

File "C:\Users\sancy\Anaconda3\lib\multiprocessing\pool.py", line 683, in get
raise self._value

NameError: name 'videos_root' is not defined`

Segmentation fault (core dump) on image writing back to the disk.

import imageio ###conda install -c conda-forge imageio
replace all cv2.imwrite() or scipy.misc.imsave() to imageio.imwrite()
This is a mutex issue in new opencv releases. Always use imageio to write images back to the disk.

Only 12813 for flow and 13190 for rgb in result

Heyyyy!

As known, there are 13320 videos in UCF dataset in total, but I only get 12813 results for flow extarction and 13190 for RGB frame.

is any other come across same problem?

[SOLUTIONS] Insights for setting up

I was setting this up in the end of August, 2019. Here are some findings that you need to do/change to make this work:

• scipy.misc.imsave() has been deprecated (in SciPy 1.0.0), instead, use import imageio and imageio.imwrite()
• cv2.createOptFlow_DualTVL1() did not work with opencv 4.X.X (pip install opencv-python). If installed this way, remove it (pip uninstall opencv-python) and install its different version (pip install opencv-contrib-python). Next, change cv2.createOptFlow_DualTVL1() to dtvl1=cv2.optflow.DualTVL1OpticalFlow_create()
• I had problems with the skvideo.io.vread() function. I had both ffmpeg/ffprobe installed using pip, but that did not work. I had success with conda install -c mrinaljain17 ffmpeg, but you can also install it from source.
• As I had a different dataset setup than the author (I simply extracted UCF101 dataset, so my paths looked like this: ../../UCF-101/ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01.avi, which I believe is the norm). Due to this, I had to remove the second argument for os.path.join() function found right after the main function and args = ... For example, the first line became data_root=os.path.join(args.data_root)
• If you are not parsing any arguments, change the default value for data_root. For example, parser.add_argument('--data_root',default='../../Downloads/UCF-101',type=str)

Hope this helps. Thank you for the scripts.

How long it takes to run on ucf101?

The first four among 101 categories hasn't been finished for 6 hours. Is it too slow?

What is parameter 'bound'

Hi,
I was wondering for what the parameter 'bound' exactly is and why you store x and y of the flow separately?
The i3d I use needs flows in the array size for example (224, 224, 2), are the frames merged later?
Sounds like most of you have the same intention.
I would be very happy if someone could help me out of this gap :)

Thanks for help in advance

how to speed up by gpu?

TVL1 didn't work well on moving-mnist dataset

Hi,
I am using TVL1 to capture optical flow on moving-mnist dataset (a digit moving randomly in a frame). But I found that it fail to capture optical flow. Any suggestions about it?
TVL1 flow:

image: motion is small, digit is moving vertically

No attribute error.

First of all, that code is very useful. Thanks a lot for that.
Secondly, I had a look over your code. I am implementing Video Classification task over a data set named UCF101.
I was able to store the frames successfully. However, it throws error while creating an instance of DualOpticalFlow_TVL1 module.

Module 'cv2.cv2' has no attribute 'createOptFlow_DualTVL1'

I have all the packages installed and updated. Did you face this error before? If yes, how did you resolve it?