Generating black frames about dense-flow

[agethen]

Hi,
sorry for the late reply.

Given the original code has the same problem, could something be wrong with your input file? For example, the codec might not be installed or similar.

I would suggest to first only extract images using the --imgFile="" option, and check if they look correct.

[MaAo]

Hi,

When I execute "./denseFlow_gpu --vidFile="video0.mp4" --xFlowFile="flow_x" --yFlowFile="flow_y" --imgFile="im" --bound=16 --type=2 --device_id=0 --step=10"

The return is "Unable to stop the stream: Function not implemented
Could not open video video0.mp4
Processing file: video0.mp4 (0 frames)"

Can you help me solve this?

And when I execute "make"，the result is:
"g++ -std=c++11 -Wfatal-errors -DSERIALIZE_BUFFER -I/usr/local/opencv/include -Iinclude/ -c src/dense_flow_gpu.cc
g++ -std=c++11 -Wfatal-errors -DSERIALIZE_BUFFER -I/usr/local/opencv/include -Iinclude/ -c src/toolbox.cc
g++ -o build/denseFlow_gpu dense_flow_gpu.o toolbox.o -L/usr/local/lib -lopencv_core -lopencv_cudaoptflow -lopencv_imgcodecs -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lboost_serialization
rm *.o"

Did I compile successful?

Very thanks!

[agethen]

Dear MaAo,

looks like OpenCV was not able to open the file for some reason. I assume the path to the file is correct? Try an absolute path to be certain.
What system environment are you using? Which version of OpenCV do you use? I also wonder if it is a codec problem. Can you run "ffprobe video0.mp4" ?

[MaAo]

Very thanks for reply!
I use CentOS 7 and opencv 3.3.1.
I separatly download ffmpeg which contained "ffprobe" and I can run "ffprobe video0.mp4".
I doubt that I didn't modify "INCLUDE=-I/home/phd/01/agethen/local/libopencv/include -Iinclude/" in "Makefile.config" . Can you tell me what does this path mean?

I set "INCLUDE=-I/usr/local/opencv/include -Iinclude/" . And the folders in "/usr/local/opencv/include" are "opencv" and "opencv2" .

[agethen]

The path should point to OpenCV 3 installation. It needs to be compiled with GPU support, so usually you have to download OpenCV 3 and build it first (and specify the installation path).

Recently, linux distributions started to ship with OpenCV 3, in which case you don't need to specify anything (because it will be in the default search path). BTW, I am not sure if they already include GPU support.

Sorry about being unclear, I meant: Copy & Paste the output of ffprobe. It should show what codec is being used in the video.

[MaAo]

I run "ffprobe video0.mp4" and the result is ：
[maao@f5c93f0bb4ed dense_flow_agethen]$ /nfs/project/maao/codes/tools/ffmpeg/ffprobe video0.mp4
ffprobe version 4.0.2-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2007-2018 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.12.100
Duration: 00:00:10.01, start: 0.000000, bitrate: 617 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480, 485 kb/s, 20.48 fps, 90k tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler

And I have downloaded OpenCV3 and build it . After I run "make" in your codes,I can get "denseFlow_gpu" in the folder "build" .But When I execute "./denseFlow_gpu --vidFile="video0.mp4" --xFlowFile="flow_x" --yFlowFile="flow_y" --imgFile="im" --bound=16 --type=2 --device_id=0 --step=10"
The return is "Unable to stop the stream: Function not implemented
Could not open video video0.mp4
Processing file: video0.mp4 (0 frames)"

How can I do？

[agethen]

So the codec of the video is h264, very standard.
Sorry to ask again, but are you sure the video file is also in the build folder?
Because from your ffprobe output: [maao@f5c93f0bb4ed dense_flow_agethen]$ /nfs/project/maao/codes/tools/ffmpeg/ffprobe video0.mp4
It looks like the file is in dense_flow_agethen and not in the subfolder build/ ;)
Try using the full absolute path to the video when running (an absolute path looks like this: /home/maao/dense_flow_agethen/video0.mp4).

[MaAo]

I've put the video in the subfolder build/ as you said and the result was like this :

[maao@f5c93f0bb4ed dense_flow_agethen]$ cd build
[maao@f5c93f0bb4ed build]$ ll
-rwxrwxr-x 1 maao maao 328232 Oct 16 16:25 denseFlow_gpu
-rw-rw-r-- 1 maao maao 772583 Oct 16 16:13 video0.mp4
[maao@f5c93f0bb4ed build]$ ./denseFlow_gpu --vidFile="video0.mp4" --xFlowFile="flow_x" --yFlowFile="flow_y" --imgFile="im" --bound=16 --type=2 --device_id=0 --step=10
(DEBUG) try_init_v4l2 VIDIOC_QUERYCAP "video0.mp4": Function not implemented
Unable to stop the stream: Function not implemented
Could not open video video0.mp4
Processing file: video0.mp4 (0 frames)

Do you know what the problem is? Is there a problem in my "denseFlow_gpu"?

[agethen]

No, the problem is in OpenCV or the codecs it uses (which I think come with ffmpeg?).
This is the error message from OpenCV or the codec:

(DEBUG) try_init_v4l2 VIDIOC_QUERYCAP "video0.mp4": Function not implemented
Unable to stop the stream: Function not implemented

denseFlow_gpu only prints this error: Could not open video video0.mp4.
You could try to use opencv in python to open the file:

python
import cv2
video = cv2.VideoCapture("video0.mp4")
ret, frame = video.read()

and I would guess that will also fail.

I am not sure what is wrong, I would suggest to first reinstall ffmpeg properly (I dont know how to do it in CentOS; for Debian based systems it is just apt install ffmpeg), then recompile & reinstall OpenCV after that. You can check with that python script if OpenCV is working correctly :)

[MaAo]

Hi,I tried run the the tutorial from wanglimin https://github.com/wanglimin/dense_flow
I use CentOS 7 and opencv-2.4.13.
When I run "make" ,I got this--

[maao@f5c93f0bb4ed build]$ make
Scanning dependencies of target denseFlow_gpu
[ 50%] Building CXX object CMakeFiles/denseFlow_gpu.dir/denseFlow_gpu.cpp.o
[100%] Linking CXX executable denseFlow_gpu
/usr/bin/ld: cannot find -lopencv_dep_cudart
/usr/bin/ld: cannot find -lopencv_dep_nppial
/usr/bin/ld: cannot find -lopencv_dep_nppicc
/usr/bin/ld: cannot find -lopencv_dep_nppicom
/usr/bin/ld: cannot find -lopencv_dep_nppidei
/usr/bin/ld: cannot find -lopencv_dep_nppif
/usr/bin/ld: cannot find -lopencv_dep_nppig
/usr/bin/ld: cannot find -lopencv_dep_nppim
/usr/bin/ld: cannot find -lopencv_dep_nppist
/usr/bin/ld: cannot find -lopencv_dep_nppisu
/usr/bin/ld: cannot find -lopencv_dep_nppitc
collect2: error: ld returned 1 exit status
make[2]: *** [denseFlow_gpu] Error 1
make[1]: *** [CMakeFiles/denseFlow_gpu.dir/all] Error 2
make: *** [all] Error 2

Can you help me feed this?
Very thanks!

[agethen]

Sorry, I can't help you for the OpenCV 2 version, but maybe wanglimin can. I can only tell you that I don't have any such files in my system's OpenCV 2 installation.