yjxiong / caffe Goto Github PK

Hi @yjxiong,
I tried to finetune the spatial-VGG-16 network with your prototxt file and your RGB frames split. However, I met the errors cannot load file. I checked that my data is alright with your format %04.jpg starting index from 1. (I modified your code a little bit at https://github.com/yjxiong/caffe/blob/action_recog/src/caffe/util/io.cpp#L316)

1211 22:43:31.502714  4785 layer_factory.hpp:74] Creating layer data
I1211 22:43:31.502755  4785 net.cpp:94] Creating Layer data
I1211 22:43:31.502771  4785 net.cpp:413] data -> data
I1211 22:43:31.502806  4785 net.cpp:413] data -> label_local
I1211 22:43:31.502821  4785 net.cpp:158] Setting up data
I1211 22:43:31.503285  4785 video_data_layer.cpp:34] Opening file: train_rgb_split1.txt
I1211 22:43:31.509351  4785 video_data_layer.cpp:50] A total of 9537 videos.
E1211 22:43:31.509426  4785 io.cpp:320] Could not load file /home/tranlaman/Public/data/video/UCF101/WallPushups/v_WallPushups_g21_c06
F1211 22:43:31.509577  4785 video_data_layer.cpp:66] Check failed: ReadSegmentRGBToDatum(lines_[lines_id_].first, lines_[lines_id_].second, offsets, new_height, new_width, new_length, &datum, true) 
*** Check failure stack trace: ***
    @     0x7fc55c4bbdaa  (unknown)
    @     0x7fc55c4bbce4  (unknown)
    @     0x7fc55c4bb6e6  (unknown)
    @     0x7fc55c4be687  (unknown)
    @     0x7fc55c90d8ff  caffe::VideoDataLayer<>::DataLayerSetUp()
    @     0x7fc55c8ed3e5  caffe::BasePrefetchingDataLayer<>::LayerSetUp()
    @     0x7fc55c83ad02  caffe::Net<>::Init()
    @     0x7fc55c83c963  caffe::Net<>::Net()
    @     0x7fc55c9241c9  caffe::Solver<>::InitTrainNet()
    @     0x7fc55c92526e  caffe::Solver<>::Init()
    @     0x7fc55c925406  caffe::Solver<>::Solver()
    @           0x417f40  caffe::GetSolver<>()
    @           0x40ff02  train()
    @           0x40e6e6  main
    @     0x7fc55af75ec5  (unknown)
    @           0x40eccb  (unknown)
    @              (nil)  (unknown)
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 4786 on node uCom exited on signal 6 (Aborted).
--------------------------------------------------------------------------

Thanks,

Hi, @yjxiong
I use your matlab test code with the temporal caffe model you provided to test the accuray of UCF split1,2,3
The result is 84%,86.2%,85.5%.
In your paper the result is 85.7%,88.2%,87.4%
I use your denseflow_gpu code to extrace optical flow like this './denseFlow_gpu -f myvideo.avi -x tmp/flow_x -y tmp/flow_y -i tmp/image -b 20 -t 1 -d 0 -s 1 '

@wanglimin
when I run the 'extractOpticalFlow.m',it occured :

/bin/bash: ./extract_gpu: No such file or directory
......
cmd = sprintf('./extract_gpu -f %s -x %s -y %s -i %s -b 20',file1,file2,file3,file4);
system(cmd);
end

I tried to addpath ,but it's no use.look forward to your help.Thanks.

Hi @yjxiong,
I am a problem with speed of running cuhk-caffe. I update caffe with your repo few days ago, and I build Caffe successfully, but running speed at testing time is quite slow. It is slower than previous version in December 2015. One thing that I notice is that there are many messages from Caffe as bellow:

I0202 15:18:48.506283 21248 cudnn_conv_layer.cpp:179] Optimized cudnn conv

I hope to receive some feedback from you.
Thank you.

您好。感谢你对多机多卡caffe的共享。目前我在看源码的时候遇到一个小小的困惑，就是在net.cpp中的BackwardFromTo方法中，完成backward运算之后需要对所有GPU的梯度做一个合并操作，即代码“caffe_iallreduce(this->params_[n]->mutable_cpu_diff(),this->params_[n]->count());”我的疑问是改代码中为什么会使用cpu_diff来合并，运算过程中不是把梯度存在gpu_diff中吗？没有看到有哪个地方是吧gpu_dff拷贝到cpu_diff中的。期盼您的解答，谢谢！

Hi, @yjxiong
I'm using your openmpi based caffe for parallel training. I noticed that when I was running examples of mnist on 2 nodes in a parallel manner, the train speed was much slower than that of just 1 node. The "Model Synchronization Communication time " was 0.14s or so.
In the lenet_parallel_solver.prototxt, I just modified device_id: [0, 0](cause I have only 1 GPU on each node).
And my openmpi-1.8.8 was built with CUDA-aware support.
My 2 nodes was connected using SSH.
Is there something wrong I had made?
Anyway, Thanks a lot for your work!

Hi @yjxiong
I have a question to ask your guys here. I run very deep two-stream cnn on my data, but I do not know why my caffe is reading images from disk very slow. Most of the time spends on reading from disk with unreasonable proportional (about 1s computing and 8s idle, might be reading images from disks as illustrated in the image). Is is an expected behavior? Or a bug? T

Best regards,

Hi, have you train very deep two stream on HMDB51 dataset?
I found it easily overfitting on this dataset even using corner cropping and multi-scale cropping

@wanglimin
Thanks！
Whether the RGB & Flow images created by the dense-folw tool need to rename for train the net.

Hi there!

I was testing for the first split using your optical flow data, when I noticed that there were a few folders that couldn't be extracted from the zip. Does it comprise the optical flow for all videos?

Hi, can the code run correctly with a single GPU ?
Specailly, complies with USE_MPI = off ( already modify the batch_size related parameters ). Furthermore, may I ask what server you used for training? I found our Dell R720 can't support mutiple GPUs because of the power problem.

Thanks.

`-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost version: 1.54.0
-- Found the following Boost libraries:
-- system
-- thread
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found GFlags: /usr/include
-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found Glog: /usr/include
-- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found PROTOBUF: /usr/lib/x86_64-linux-gnu/libprotobuf.so
-- Found PROTOBUF Compiler: /usr/bin/protoc
-- Found HDF5: /usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so
-- Found LMDB: /usr/include
-- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB: /usr/include
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy: /usr/include
-- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
-- CUDA detected: 7.0
-- Found cuDNN (include: /usr/local/include, library: /usr/local/lib/libcudnn.so)
-- Added CUDA NVCC flags for: sm_35
-- OpenCV found (/usr/local/share/OpenCV)
-- Found Atlas: /usr/include
-- Found Atlas (include: /usr/include, library: /usr/lib/libatlas.so)
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.6", minimum required is "2.7")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.6", minimum required is "2.7")
-- Found NumPy: /usr/lib/python2.7/dist-packages/numpy/core/include (found suitable version "1.8.2", minimum required is "1.7.1")
-- NumPy ver. 1.8.2 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include)
-- Could NOT find Boost
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.6")
-- Found MPI_C: /usr/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
-- Found MPI_CXX: /usr/lib/libmpi_cxx.so;/usr/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
-- Python interface is disabled or not all required dependecies found. Building without it...
-- Detected Doxygen OUTPUT_DIRECTORY: ./doxygen/
-- Found Git: /usr/bin/git (found version "1.9.1")

-- Caffe Configuration Summary
-- General:
-- Version : (Caffe doesn't declare its version in headers)
-- Git : v0.9999-1553-gc54adf0-dirty
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Build type : Release

-- BUILD_SHARED_LIBS : ON
-- BUILD_python : ON
-- BUILD_matlab : OFF
-- BUILD_docs : ON
-- CPU_ONLY : OFF

-- Dependencies:
-- BLAS : Yes (Atlas)
-- Boost : Yes (ver. 1.54)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 2.5.0)
-- lmdb : Yes (ver. 0.9.10)
-- Snappy : Yes (ver. 1.1.0)
-- LevelDB : Yes (ver. 1.15)
-- OpenCV : Yes (ver. 2.4.12)
-- CUDA : Yes (ver. 7.0)

-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_35
-- cuDNN : Yes

-- Documentaion:
-- Doxygen : /usr/bin/doxygen (1.8.6)
-- config_file : /home/zyy/workspace/wangml/caffe-action/.Doxyfile

-- Install:
-- Install path : /home/zyy/workspace/wangml/caffe-action/build/install

-- Configuring done
-- Generating done
-- Build files have been written to: /home/zyy/workspace/wangml/caffe-action/build
`

@wanglimin
作者你好，感谢你分享你的代码，我编译的时候遇到了一些问题，但是一直没解决，希望您能抽时间帮我看看，不胜感激！
运行环境：
openmpi 1.8.8
python 2.7
matlab 2014a
opencv 2.4.12
上面是我cmake .. -DUSE_MPI=ON得到的结果，涂黑的地方是我的问题。1、Boost，后面编译都知道我装的是1.5.4版本，为什么前面会提示找不到呢？这个会不会对后续有影响？2、python和matlab编译，因为我对cmake不太熟，我没太明白怎么用cmake手动编译matcaffe和pycaffe。而且因为matcaffe和pycaffe都没有编译成功，所以后面我运行你的两个demo的时候，都出现了找不到caffe的情况。

After running train_action_recognition_flow.sh with the optical flow data and the pre-trained model of the website, I was not able to achieve the 86% for vgg_16 of the paper(Towards Good Practices for Very Deep Two-Stream ConvNets). I only achieved 68%, because of that I would like to know if vgg_16_flow_train_val_fast.prototxt is the correct prototxt of the paper.

在VideoData层中，训练数据是这样的：
ucf101_rgb_img/WallPushups/v_WallPushups_g21_c06 111 98，
111也就是从这个视频提取了111 frames（RGB或者optical flow），
我想了解下这个层的一些参数是什么意思。
new_length、new_segments表示什么？
特别是batch_size指的是一个iteration的frames的数目？还是指的video的数目？拿训练数据train_rgb_split1.txt举例来说就是：batch_size是指的111中的多少幅图像，还是train_rgb_split1.txt中的batch_size数目的行数？
另外想问下要做real-time的action recognition，您有什么建议么？
希望您能回答一下，谢谢了。

Hi guys,

When I try to adapt the code for HMDB51 dataset (basically follow all the settings in the UCF101 sample project), I came across the following error:

I0201 13:56:47.363147 17515 layer_factory.hpp:74] Creating layer data
I0201 13:56:47.363158 17515 net.cpp:113] Creating Layer data
I0201 13:56:47.363162 17515 net.cpp:432] data -> data
I0201 13:56:47.363180 17515 net.cpp:432] data -> label_local
I0201 13:56:47.363185 17515 net.cpp:177] Setting up data
I0201 13:56:47.363440 17515 video_data_layer.cpp:34] Opening file: ~/hmdb51/train_rgb_split1.txt
I0201 13:56:47.364840 17515 video_data_layer.cpp:50] A total of 3570 videos.
I0201 13:56:47.368875 17515 video_data_layer.cpp:76] output data size: 16,3,224,224
I0201 13:56:47.369961 17515 net.cpp:184] Top shape: 16 3 224 224 (2408448)
I0201 13:56:47.369994 17515 net.cpp:184] Top shape: 16 1 1 1 (16)
I0201 13:56:47.370004 17515 layer_factory.hpp:74] Creating layer conv1_1
I0201 13:56:47.370019 17515 net.cpp:113] Creating Layer conv1_1
I0201 13:56:47.370025 17515 net.cpp:476] conv1_1 <- data
I0201 13:56:47.370033 17515 net.cpp:144] This layer is inheriting previous layer's sync mode: 1
I0201 13:56:47.370040 17515 net.cpp:432] conv1_1 -> conv1_1
I0201 13:56:47.370061 17515 net.cpp:177] Setting up conv1_1
*** Aborted at 1454363807 (unix time) try "date -d @1454363807" if you are using GNU date ***
PC: @     0x7f7fb9bfa73e caffe::VideoDataLayer<>::InternalThreadEntry()
*** SIGFPE (@0x7f7fb9bfa73e) received by PID 17516 (TID 0x7f7f714ce700) from PID 18446744072530929470; stack trace: ***
    @     0x7f7fb85ae2f0 (unknown)
    @     0x7f7fb9bfa73e caffe::VideoDataLayer<>::InternalThreadEntry()
    @     0x7f7fb7e6009a (unknown)
    @     0x7f7fb7c3c6aa start_thread
    @     0x7f7fb867feed (unknown)
    @                0x0 (unknown)
*** Aborted at 1454363807 (unix time) try "date -d @1454363807" if you are using GNU date ***
PC: @     0x7f23fbef1783 (unknown)
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 17516 on node yzhu25 exited on signal 8 (Floating point exception).

As there were not much debug information, could anyone point out where maybe the issue? For the floating point exception, I checked online and it seems other people using Caffe also faced with it before. It kind of relates to cropping size is larger than height or width, or the mirror operation. But I don't know how to deal with it here. Thanks a lot.

Hi, @wanglimin @yjxiong

I use caffe test commond to test the temporal model you provided.

~/caffe-action/build/install/bin/caffe test -model vgg_16_flow_train_val_fast.prototxt -weights ~/very-deep-two-stream/wanglimin-model/cuhk_action_temporal_vgg_16_split1.caffemodel   -gpu 0

But the accuray all zero. I can't find what's wrong (but the matlab test code works).

I0831 20:58:21.425902  5977 caffe.cpp:210] Running for 50 iterations.
I0831 20:58:21.705755  5977 caffe.cpp:234] Batch 0, accuracy = 0
I0831 20:58:21.705826  5977 caffe.cpp:234] Batch 0, loss = 4.56581
I0831 20:58:21.842283  5977 caffe.cpp:234] Batch 1, accuracy = 0
I0831 20:58:21.842308  5977 caffe.cpp:234] Batch 1, loss = 4.57135
I0831 20:58:21.968304  5977 caffe.cpp:234] Batch 2, accuracy = 0
I0831 20:58:21.968328  5977 caffe.cpp:234] Batch 2, loss = 4.56923
I0831 20:58:22.085276  5977 caffe.cpp:234] Batch 3, accuracy = 0
I0831 20:58:22.085300  5977 caffe.cpp:234] Batch 3, loss = 4.56122
I0831 20:58:22.200157  5977 caffe.cpp:234] Batch 4, accuracy = 0.2
I0831 20:58:22.200182  5977 caffe.cpp:234] Batch 4, loss = 4.55828
I0831 20:58:22.312468  5977 caffe.cpp:234] Batch 5, accuracy = 0
I0831 20:58:22.312490  5977 caffe.cpp:234] Batch 5, loss = 4.55105
I0831 20:58:22.422643  5977 caffe.cpp:234] Batch 6, accuracy = 0
I0831 20:58:22.422667  5977 caffe.cpp:234] Batch 6, loss = 4.58159
I0831 20:58:22.533643  5977 caffe.cpp:234] Batch 7, accuracy = 0
I0831 20:58:22.533668  5977 caffe.cpp:234] Batch 7, loss = 4.59634
I0831 20:58:22.646528  5977 caffe.cpp:234] Batch 8, accuracy = 0
I0831 20:58:22.646553  5977 caffe.cpp:234] Batch 8, loss = 4.59047
I0831 20:58:22.761013  5977 caffe.cpp:234] Batch 9, accuracy = 0
I0831 20:58:22.761052  5977 caffe.cpp:234] Batch 9, loss = 4.62319
I0831 20:58:22.877241  5977 caffe.cpp:234] Batch 10, accuracy = 0
I0831 20:58:22.877264  5977 caffe.cpp:234] Batch 10, loss = 4.63062
I0831 20:58:22.992969  5977 caffe.cpp:234] Batch 11, accuracy = 0
I0831 20:58:22.992990  5977 caffe.cpp:234] Batch 11, loss = 4.63736
I0831 20:58:23.105216  5977 caffe.cpp:234] Batch 12, accuracy = 0
I0831 20:58:23.105239  5977 caffe.cpp:234] Batch 12, loss = 4.64331
I0831 20:58:23.216982  5977 caffe.cpp:234] Batch 13, accuracy = 0
I0831 20:58:23.217003  5977 caffe.cpp:234] Batch 13, loss = 4.63844
I0831 20:58:23.328269  5977 caffe.cpp:234] Batch 14, accuracy = 0
I0831 20:58:23.328290  5977 caffe.cpp:234] Batch 14, loss = 4.62079
I0831 20:58:23.436784  5977 caffe.cpp:234] Batch 15, accuracy = 0
I0831 20:58:23.436825  5977 caffe.cpp:234] Batch 15, loss = 4.59784
I0831 20:58:23.547108  5977 caffe.cpp:234] Batch 16, accuracy = 0
I0831 20:58:23.547130  5977 caffe.cpp:234] Batch 16, loss = 4.60577
I0831 20:58:23.658761  5977 caffe.cpp:234] Batch 17, accuracy = 0
I0831 20:58:23.658782  5977 caffe.cpp:234] Batch 17, loss = 4.59781
I0831 20:58:23.770740  5977 caffe.cpp:234] Batch 18, accuracy = 0
I0831 20:58:23.770763  5977 caffe.cpp:234] Batch 18, loss = 4.59607
I0831 20:58:23.882455  5977 caffe.cpp:234] Batch 19, accuracy = 0
I0831 20:58:23.882477  5977 caffe.cpp:234] Batch 19, loss = 4.60705
I0831 20:58:23.993892  5977 caffe.cpp:234] Batch 20, accuracy = 0
I0831 20:58:23.993913  5977 caffe.cpp:234] Batch 20, loss = 4.60775
I0831 20:58:24.102294  5977 caffe.cpp:234] Batch 21, accuracy = 0
I0831 20:58:24.102314  5977 caffe.cpp:234] Batch 21, loss = 4.59202
I0831 20:58:24.209605  5977 caffe.cpp:234] Batch 22, accuracy = 0
I0831 20:58:24.209630  5977 caffe.cpp:234] Batch 22, loss = 4.60682
I0831 20:58:24.318430  5977 caffe.cpp:234] Batch 23, accuracy = 0
I0831 20:58:24.318454  5977 caffe.cpp:234] Batch 23, loss = 4.62756
I0831 20:58:24.424371  5977 caffe.cpp:234] Batch 24, accuracy = 0
I0831 20:58:24.424392  5977 caffe.cpp:234] Batch 24, loss = 4.61974
I0831 20:58:24.530743  5977 caffe.cpp:234] Batch 25, accuracy = 0
I0831 20:58:24.530766  5977 caffe.cpp:234] Batch 25, loss = 4.62237
I0831 20:58:24.637039  5977 caffe.cpp:234] Batch 26, accuracy = 0
I0831 20:58:24.637059  5977 caffe.cpp:234] Batch 26, loss = 4.6219
I0831 20:58:24.744755  5977 caffe.cpp:234] Batch 27, accuracy = 0
I0831 20:58:24.744776  5977 caffe.cpp:234] Batch 27, loss = 4.6247
I0831 20:58:24.852677  5977 caffe.cpp:234] Batch 28, accuracy = 0
I0831 20:58:24.852699  5977 caffe.cpp:234] Batch 28, loss = 4.61173
I0831 20:58:24.960541  5977 caffe.cpp:234] Batch 29, accuracy = 0
I0831 20:58:24.960561  5977 caffe.cpp:234] Batch 29, loss = 4.6213
I0831 20:58:25.068442  5977 caffe.cpp:234] Batch 30, accuracy = 0
I0831 20:58:25.068462  5977 caffe.cpp:234] Batch 30, loss = 4.60979
I0831 20:58:25.175941  5977 caffe.cpp:234] Batch 31, accuracy = 0
I0831 20:58:25.175961  5977 caffe.cpp:234] Batch 31, loss = 4.60837
I0831 20:58:25.282083  5977 caffe.cpp:234] Batch 32, accuracy = 0
I0831 20:58:25.282104  5977 caffe.cpp:234] Batch 32, loss = 4.61826
I0831 20:58:25.388100  5977 caffe.cpp:234] Batch 33, accuracy = 0
I0831 20:58:25.388120  5977 caffe.cpp:234] Batch 33, loss = 4.61304
I0831 20:58:25.494299  5977 caffe.cpp:234] Batch 34, accuracy = 0
I0831 20:58:25.494320  5977 caffe.cpp:234] Batch 34, loss = 4.60632
I0831 20:58:25.600524  5977 caffe.cpp:234] Batch 35, accuracy = 0
I0831 20:58:25.600544  5977 caffe.cpp:234] Batch 35, loss = 4.60945
I0831 20:58:25.706799  5977 caffe.cpp:234] Batch 36, accuracy = 0
I0831 20:58:25.706820  5977 caffe.cpp:234] Batch 36, loss = 4.6146
I0831 20:58:25.812988  5977 caffe.cpp:234] Batch 37, accuracy = 0
I0831 20:58:25.813009  5977 caffe.cpp:234] Batch 37, loss = 4.63083
I0831 20:58:25.919095  5977 caffe.cpp:234] Batch 38, accuracy = 0
I0831 20:58:25.919114  5977 caffe.cpp:234] Batch 38, loss = 4.58947
I0831 20:58:26.025218  5977 caffe.cpp:234] Batch 39, accuracy = 0
I0831 20:58:26.025239  5977 caffe.cpp:234] Batch 39, loss = 4.60846
I0831 20:58:26.133335  5977 caffe.cpp:234] Batch 40, accuracy = 0
I0831 20:58:26.133358  5977 caffe.cpp:234] Batch 40, loss = 4.61726
I0831 20:58:26.241446  5977 caffe.cpp:234] Batch 41, accuracy = 0
I0831 20:58:26.241468  5977 caffe.cpp:234] Batch 41, loss = 4.60415
I0831 20:58:26.349599  5977 caffe.cpp:234] Batch 42, accuracy = 0
I0831 20:58:26.349620  5977 caffe.cpp:234] Batch 42, loss = 4.62642
I0831 20:58:26.457484  5977 caffe.cpp:234] Batch 43, accuracy = 0
I0831 20:58:26.457504  5977 caffe.cpp:234] Batch 43, loss = 4.61726
I0831 20:58:26.565490  5977 caffe.cpp:234] Batch 44, accuracy = 0
I0831 20:58:26.565513  5977 caffe.cpp:234] Batch 44, loss = 4.62254
I0831 20:58:26.674124  5977 caffe.cpp:234] Batch 45, accuracy = 0
I0831 20:58:26.674144  5977 caffe.cpp:234] Batch 45, loss = 4.61613
I0831 20:58:26.781986  5977 caffe.cpp:234] Batch 46, accuracy = 0
I0831 20:58:26.782007  5977 caffe.cpp:234] Batch 46, loss = 4.62502
I0831 20:58:26.888824  5977 caffe.cpp:234] Batch 47, accuracy = 0
I0831 20:58:26.888845  5977 caffe.cpp:234] Batch 47, loss = 4.58574
I0831 20:58:26.995069  5977 caffe.cpp:234] Batch 48, accuracy = 0
I0831 20:58:26.995091  5977 caffe.cpp:234] Batch 48, loss = 4.60927
I0831 20:58:27.101228  5977 caffe.cpp:234] Batch 49, accuracy = 0.2
I0831 20:58:27.101250  5977 caffe.cpp:234] Batch 49, loss = 4.60648
I0831 20:58:27.101258  5977 caffe.cpp:239] Loss: 4.60677
I0831 20:58:27.101277  5977 caffe.cpp:251] accuracy = 0.008
I0831 20:58:27.101294  5977 caffe.cpp:251] loss = 4.60677 (* 1 = 4.60677 loss)

Hi yjxiong,

I am new to Caffe, and I am trying to download the pre-trained models on this page, but I received an error 404 not found.

I followed the instructions on the Caffe Model Zoo and tried the following command in your project directory:

./scripts/download_model_from_gist.sh d26b3b8b8eec182a27ce9871752fedd374b63650 ./models/action_recognition/

Output:

Downloading Caffe model info to ./models/action_recognition//d26b3b8b8eec182a27ce9871752fedd374b63650 ...
--2016-03-08 16:43:12-- https://gist.github.com/d26b3b8b8eec182a27ce9871752fedd374b63650/download
....
[unnecessary error info messages]
....
Proxy request sent, awaiting response... 404 Not Found
2016-03-08 16:43:13 ERROR 404: Not Found.

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Done

Is the git hash for the spatial and temporal models incorrect? Or have I used the incorrect commands?
Thank you in advance!

In vgg_16_flow_train_val_fast.prototxt, the net name is : "CUHK_Action_SpatialNet".
In vgg_16_rgb_train_val_fast.prototxt, the net name is : "CUHK_Action_TemporalNet".

From your arxiv paper, I read this "The spatial net is built on a single frame image (224×224×3) and therefore its architecture is the same as those for object recognition in image domain. The input of temporal net is 10-frame stacking of optical flow fields (224×224×20)."

Are they just simple typos? Or I misunderstood something here. Thanks.

After running make runtest I get:

[==========] Running 1356 tests from 214 test cases.
[----------] Global test environment set-up.
[----------] 27 tests from ReductionLayerTest/1, where TypeParam = caffe::CPUDevice
[ RUN ] ReductionLayerTest/1.TestAbsSumCoeffAxis1Gradient
[ OK ] ReductionLayerTest/1.TestAbsSumCoeffAxis1Gradient (1 ms)
[ RUN ] ReductionLayerTest/1.TestSum
[ OK ] ReductionLayerTest/1.TestSum (0 ms)
[ RUN ] ReductionLayerTest/1.TestMean
[ OK ] ReductionLayerTest/1.TestMean (0 ms)
[ RUN ] ReductionLayerTest/1.TestAbsSum
[ OK ] ReductionLayerTest/1.TestAbsSum (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumCoeffAxis1
[ OK ] ReductionLayerTest/1.TestSumCoeffAxis1 (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumGradient
[ OK ] ReductionLayerTest/1.TestSumGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquaresCoeffGradient
[ OK ] ReductionLayerTest/1.TestSumOfSquaresCoeffGradient (1 ms)
[ RUN ] ReductionLayerTest/1.TestMeanCoeffAxis1
[ OK ] ReductionLayerTest/1.TestMeanCoeffAxis1 (0 ms)
[ RUN ] ReductionLayerTest/1.TestAbsSumCoeff
[ OK ] ReductionLayerTest/1.TestAbsSumCoeff (0 ms)
[ RUN ] ReductionLayerTest/1.TestAbsSumCoeffAxis1
[ OK ] ReductionLayerTest/1.TestAbsSumCoeffAxis1 (0 ms)
[ RUN ] ReductionLayerTest/1.TestMeanCoeffGradient
[ OK ] ReductionLayerTest/1.TestMeanCoeffGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquaresCoeffAxis1Gradient
[ OK ] ReductionLayerTest/1.TestSumOfSquaresCoeffAxis1Gradient (1 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquaresCoeff
[ OK ] ReductionLayerTest/1.TestSumOfSquaresCoeff (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumCoeff
[ OK ] ReductionLayerTest/1.TestSumCoeff (0 ms)
[ RUN ] ReductionLayerTest/1.TestSetUpWithAxis1
[ OK ] ReductionLayerTest/1.TestSetUpWithAxis1 (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquaresGradient
[ OK ] ReductionLayerTest/1.TestSumOfSquaresGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestAbsSumGradient
[ OK ] ReductionLayerTest/1.TestAbsSumGradient (1 ms)
[ RUN ] ReductionLayerTest/1.TestMeanCoeff
[ OK ] ReductionLayerTest/1.TestMeanCoeff (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumCoeffGradient
[ OK ] ReductionLayerTest/1.TestSumCoeffGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquaresCoeffAxis1
[ OK ] ReductionLayerTest/1.TestSumOfSquaresCoeffAxis1 (0 ms)
[ RUN ] ReductionLayerTest/1.TestMeanCoeffGradientAxis1
[ OK ] ReductionLayerTest/1.TestMeanCoeffGradientAxis1 (1 ms)
[ RUN ] ReductionLayerTest/1.TestMeanGradient
[ OK ] ReductionLayerTest/1.TestMeanGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumOfSquares
[ OK ] ReductionLayerTest/1.TestSumOfSquares (0 ms)
[ RUN ] ReductionLayerTest/1.TestSumCoeffAxis1Gradient
[ OK ] ReductionLayerTest/1.TestSumCoeffAxis1Gradient (1 ms)
[ RUN ] ReductionLayerTest/1.TestAbsSumCoeffGradient
[ OK ] ReductionLayerTest/1.TestAbsSumCoeffGradient (0 ms)
[ RUN ] ReductionLayerTest/1.TestSetUpWithAxis2
[ OK ] ReductionLayerTest/1.TestSetUpWithAxis2 (0 ms)
[ RUN ] ReductionLayerTest/1.TestSetUp
[ OK ] ReductionLayerTest/1.TestSetUp (0 ms)
[----------] 27 tests from ReductionLayerTest/1 (6 ms total)

[----------] 2 tests from EuclideanLossLayerTest/0, where TypeParam = caffe::CPUDevice
[ RUN ] EuclideanLossLayerTest/0.TestForward
[ OK ] EuclideanLossLayerTest/0.TestForward (1 ms)
[ RUN ] EuclideanLossLayerTest/0.TestGradient
[ OK ] EuclideanLossLayerTest/0.TestGradient (0 ms)
[----------] 2 tests from EuclideanLossLayerTest/0 (1 ms total)

[----------] 12 tests from ReshapeLayerTest/2, where TypeParam = caffe::GPUDevice
[ RUN ] ReshapeLayerTest/2.TestFlattenOutputSizes
[ OK ] ReshapeLayerTest/2.TestFlattenOutputSizes (0 ms)
[ RUN ] ReshapeLayerTest/2.TestInsertSingletonAxesStart
[ OK ] ReshapeLayerTest/2.TestInsertSingletonAxesStart (0 ms)
[ RUN ] ReshapeLayerTest/2.TestInsertSingletonAxesMiddle
[ OK ] ReshapeLayerTest/2.TestInsertSingletonAxesMiddle (0 ms)
[ RUN ] ReshapeLayerTest/2.TestInsertSingletonAxesEnd
[ OK ] ReshapeLayerTest/2.TestInsertSingletonAxesEnd (0 ms)
[ RUN ] ReshapeLayerTest/2.TestFlattenMiddle
[ OK ] ReshapeLayerTest/2.TestFlattenMiddle (0 ms)
[ RUN ] ReshapeLayerTest/2.TestForwardAfterReshape
[ OK ] ReshapeLayerTest/2.TestForwardAfterReshape (0 ms)
[ RUN ] ReshapeLayerTest/2.TestCopyDimensions
[ OK ] ReshapeLayerTest/2.TestCopyDimensions (0 ms)
[ RUN ] ReshapeLayerTest/2.TestInferenceOfUnspecified
[ OK ] ReshapeLayerTest/2.TestInferenceOfUnspecified (0 ms)
[ RUN ] ReshapeLayerTest/2.TestInferenceOfUnspecifiedWithStartAxis
[ OK ] ReshapeLayerTest/2.TestInferenceOfUnspecifiedWithStartAxis (0 ms)
[ RUN ] ReshapeLayerTest/2.TestFlattenValues
[ OK ] ReshapeLayerTest/2.TestFlattenValues (0 ms)
[ RUN ] ReshapeLayerTest/2.TestForward
[ OK ] ReshapeLayerTest/2.TestForward (0 ms)
[ RUN ] ReshapeLayerTest/2.TestGradient
[ OK ] ReshapeLayerTest/2.TestGradient (12 ms)
[----------] 12 tests from ReshapeLayerTest/2 (12 ms total)

[----------] 5 tests from Im2colLayerTest/1, where TypeParam = caffe::CPUDevice
[ RUN ] Im2colLayerTest/1.TestForward
[ OK ] Im2colLayerTest/1.TestForward (0 ms)
[ RUN ] Im2colLayerTest/1.TestRect
[ OK ] Im2colLayerTest/1.TestRect (0 ms)
[ RUN ] Im2colLayerTest/1.TestGradient
[ OK ] Im2colLayerTest/1.TestGradient (197 ms)
[ RUN ] Im2colLayerTest/1.TestRectGradient
[ OK ] Im2colLayerTest/1.TestRectGradient (178 ms)
[ RUN ] Im2colLayerTest/1.TestSetup
[ OK ] Im2colLayerTest/1.TestSetup (0 ms)
[----------] 5 tests from Im2colLayerTest/1 (375 ms total)

[----------] 5 tests from MemoryDataLayerTest/1, where TypeParam = caffe::CPUDevice
[ RUN ] MemoryDataLayerTest/1.AddMatVectorDefaultTransform
F1130 14:00:04.401077 14163 data_transformer.cpp:474] Check failed: cv_cropped_img.data
*** Check failure stack trace: ***
@ 0x7fb3f47d6d8d google::LogMessage::Fail()
@ 0x7fb3f47d8d43 google::LogMessage::SendToLog()
@ 0x7fb3f47d6909 google::LogMessage::Flush()
@ 0x7fb3f47d976e google::LogMessageFatal::~LogMessageFatal()
@ 0x7fb3f4e4c285 caffe::DataTransformer<>::Transform()
@ 0x7fb3f4e4f841 caffe::DataTransformer<>::Transform()
@ 0x7fb3f4df428b caffe::MemoryDataLayer<>::AddMatVector()
@ 0x78b341 caffe::MemoryDataLayerTest_AddMatVectorDefaultTransform_Test<>::TestBody()
@ 0xa9b353 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0xa9238a testing::Test::Run()
@ 0xa924d8 testing::TestInfo::Run()
@ 0xa925e5 testing::TestCase::Run()
@ 0xa93938 testing::internal::UnitTestImpl::RunAllTests()
@ 0xa93c13 testing::UnitTest::Run()
@ 0x7006ce main
@ 0x7fb3ed0b7a40 __libc_start_main
@ 0x702179 _start
Aborted (core dumped)
src/caffe/test/CMakeFiles/runtest.dir/build.make:49: recipe for target 'src/caffe/test/CMakeFiles/runtest' failed
make[3]: *** [src/caffe/test/CMakeFiles/runtest] Error 134
CMakeFiles/Makefile2:273: recipe for target 'src/caffe/test/CMakeFiles/runtest.dir/all' failed
make[2]: *** [src/caffe/test/CMakeFiles/runtest.dir/all] Error 2
CMakeFiles/Makefile2:281: recipe for target 'src/caffe/test/CMakeFiles/runtest.dir/rule' failed
make[1]: *** [src/caffe/test/CMakeFiles/runtest.dir/rule] Error 2
Makefile:216: recipe for target 'runtest' failed
make: *** [runtest] Error 2

It seems that the problem is related to Opencv, but I have the last version of it. That is why I would like to know if a specific version it is needed.

Hi! @wanglimin @yjxiong

When I run the action_python demo.py script , I got a problem like this:

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 538739211
I0316 11:12:52.415251 3143 cudnn_conv_layer.cpp:179] Optimized cudnn conv
Traceback (most recent call last):
File "demo.py", line 74, in
main()
File "demo.py", line 53, in main
start_frame)
File "/home/gnss/deeplearing/caffe/action_python/VideoTemporalPrediction.py", line 85, in VideoTemporalPrediction
prediction[:, span] = np.transpose(output[feature_layer])
KeyError: 'fc8-2'

Hope your help !
Thank you !

Hi @yjxiong @wanglimin

我目前只有一块4G显存的GTX 970 显卡，在跑action_python demo,py 时内存不足，想问下有解决的办法吗？后面想更换自己的数据训练，一块显卡能跑吗？

I want to replicate your results, could you provide your UCF101_rgb_image data?
Thank you.

Hi！
@wanglimin @yjxiong

前辈好！
我用vgg_16_action_flow_pretrain.caffemodel，做finetuning；训练数据集是UCF-101再加上5种行为，
网络模型只改动了fc8层的输出内别101->106;
因为我是单GPU，我删除了Gather层，batch_size = 8 ，>8会内存不足，iter_size=32
网络文件：
name: "TemporalNet"
layer {
name: "data"
type: "VideoData"
top: "data"
top: "label"
video_data_param {
source: "examples/train_flow.txt"
batch_size: 8
new_length: 10
num_segments: 1
modality: FLOW
}
transform_param{
crop_size: 224
mirror: true
fix_crop: true
multi_scale: true
max_distort: 1
scale_ratios: [1,.875,.75]
is_flow: true
mean_value: [128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128]
}
include: { phase: TRAIN }
}
layer {
name: "data"
type: "VideoData"
top: "data"
top: "label"
video_data_param {
source: "examples/val_flow.txt"
batch_size: 8
new_length: 10
num_segments: 1
modality: FLOW
}
transform_param{
crop_size: 224
mirror: false
is_flow: true
mean_value: [128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128]
}
include: { phase: TEST }
}
layer {
name: "conv1_1"
type: "Convolution"
bottom: "data"
top: "conv1_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
}
}
layer {
name: "relu1_1"
type: "ReLU"
bottom: "conv1_1"
top: "conv1_1"
}
layer {
name: "conv1_2"
type: "Convolution"
bottom: "conv1_1"
top: "conv1_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
}
}
layer {
name: "relu1_2"
type: "ReLU"
bottom: "conv1_2"
top: "conv1_2"
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1_2"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv2_1"
type: "Convolution"
bottom: "pool1"
top: "conv2_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
}
}
layer {
name: "relu2_1"
type: "ReLU"
bottom: "conv2_1"
top: "conv2_1"
}
layer {
name: "conv2_2"
type: "Convolution"
bottom: "conv2_1"
top: "conv2_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
}
}
layer {
name: "relu2_2"
type: "ReLU"
bottom: "conv2_2"
top: "conv2_2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2_2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv3_1"
type: "Convolution"
bottom: "pool2"
top: "conv3_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
}
}
layer {
name: "relu3_1"
type: "ReLU"
bottom: "conv3_1"
top: "conv3_1"
}
layer {
name: "conv3_2"
type: "Convolution"
bottom: "conv3_1"
top: "conv3_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
}
}
layer {
name: "relu3_2"
type: "ReLU"
bottom: "conv3_2"
top: "conv3_2"
}
layer {
name: "conv3_3"
type: "Convolution"
bottom: "conv3_2"
top: "conv3_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
}
}
layer {
name: "relu3_3"
type: "ReLU"
bottom: "conv3_3"
top: "conv3_3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3_3"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv4_1"
type: "Convolution"
bottom: "pool3"
top: "conv4_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu4_1"
type: "ReLU"
bottom: "conv4_1"
top: "conv4_1"
}
layer {
name: "conv4_2"
type: "Convolution"
bottom: "conv4_1"
top: "conv4_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu4_2"
type: "ReLU"
bottom: "conv4_2"
top: "conv4_2"
}
layer {
name: "conv4_3"
type: "Convolution"
bottom: "conv4_2"
top: "conv4_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu4_3"
type: "ReLU"
bottom: "conv4_3"
top: "conv4_3"
}
layer {
name: "pool4"
type: "Pooling"
bottom: "conv4_3"
top: "pool4"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv5_1"
type: "Convolution"
bottom: "pool4"
top: "conv5_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu5_1"
type: "ReLU"
bottom: "conv5_1"
top: "conv5_1"
}
layer {
name: "conv5_2"
type: "Convolution"
bottom: "conv5_1"
top: "conv5_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu5_2"
type: "ReLU"
bottom: "conv5_2"
top: "conv5_2"
}
layer {
name: "conv5_3"
type: "Convolution"
bottom: "conv5_2"
top: "conv5_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
}
}
layer {
name: "relu5_3"
type: "ReLU"
bottom: "conv5_3"
top: "conv5_3"
}
layer {
name: "pool5"
type: "Pooling"
bottom: "conv5_3"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "fc6"
type: "InnerProduct"
bottom: "pool5"
top: "fc6"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 4096
}
}
layer {
name: "relu6"
type: "ReLU"
bottom: "fc6"
top: "fc6"
}
layer {
name: "drop6"
type: "Dropout"
bottom: "fc6"
top: "fc6"
dropout_param {
dropout_ratio: 0.9
}
}
layer {
name: "fc7"
type: "InnerProduct"
bottom: "fc6"
top: "fc7"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 4096
}
}
layer {
name: "relu7"
type: "ReLU"
bottom: "fc7"
top: "fc7"
}
layer {
name: "drop7"
type: "Dropout"
bottom: "fc7"
top: "fc7"
dropout_param {
dropout_ratio: 0.8
}
}
layer {
name: "fc8-1"
type: "InnerProduct"
bottom: "fc7"
top: "fc8-1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 106
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc8-1"
bottom: "label"
include {
phase: TRAIN
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc8-1"
bottom: "label"
top: "loss"
include {
phase: TEST
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "fc8-1"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}

配置文件：

net: "gnss_model/vgg_16_flow_train_val.prototxt"
test_iter: 500
test_interval: 1000
base_lr: 0.005
lr_policy: "step"
gamma: 0.1
stepsize: 10000
display: 20
max_iter: 30000
momentum: 0.9
weight_decay: 0.0005
snapshot: 1000
snapshot_prefix: "gnss_model/gnss"
iter_size: 32
test_initialization: false

但是训练过程中一直保持：loss =87.3365，迭代500仍然没变，这个是正常现象吗？
还是我哪里出了问题？期待前辈的回复，谢谢！

老师，您好！
非常感谢您分享。我在运行action-matlab例程时遇到一些问题，希望您能抽时间看一下是我哪里操作不对。谢谢！
运行环境：Ubuntu14.04 matlabR2013b caffe
我编译好您分享的代码，用matlab文件夹内demo测试可以得到结果，但是在运行action-matlab时总是提示:Undefined variable “caffe” or class “caffe.reset_all".
我尝试了添加路径的方法还是不行。麻烦老师看下是什么问题。再次感谢！

这个工具只有CPU怎么运行呢？

The freeze option with cuDNN's batch normalization seems to be not working properly.

@Cysu please have a look if you got some spare time.

cc @wanglimin

Hi！
@wanglimin

我只有一块4G显存的显卡，batch_size=22，内存不足，我把batch_size设置为“1”，想问下，哪些参数需要更改，训练效果可接近您文章中4GPU的识别率？

非常感谢您的分享，期待您的回复！

Currently, top_label is a number. Is there a straightforward way to get multiple labels support for video_data_layer?

Hi @wanglimin,
I am running your caffe on two GPUs using OpenMPI. However, my workstation has a power problem when running your Caffe on multiple GPUs. My workstation runned for short time and restarted. I have a question for you. Is there any difference on performance, when we are running your Caffe with np=1 on 1 GPU and the original Caffe.

mpirun -np 1 ./install/bin/caffe train --solver=<Your Solver File> [--weights=<Pretrained caffemodel>]

Thanks

Hi Yuanjun Xiong,
I want to test prediction performance of very-deep-two-stream-conv. You have provided a matlab demo, so I tried to build matcaffe with your fork of caffe. Cmake interface does not build matcaffe.
When I tried to build with make interface, it failed with OpenCV. I have installed OpenCV3.0 and I built with successfully. Is it your make file does not support OpenCV3.0? Please give me a solution.
Thanks,

The error is as follows:

AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/finetune_net.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/train_net.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/test_net.bin
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/extract_features.bin] Error 1
make: *** Waiting for unfinished jobs....

Hey, I found a minor issue: Using g++ 5.2.1, with a cuda-only (no cudnn) installation and USE_CUDNN disabled, compilation failes when evaluating CUDNN_VERSION_MIN in common_layers.hpp and layer_factory.cpp. Breaking the if condition into two solves the issue :)

你好，在配置文件中的gather_layer作用是什么？只有在使用多GPU训练时才有用么？单GPU训练可以移除么？

Hi Mr. Xiong Yuanjun,
I want to report that I cannot download flow data. It failed on the halfway, and I tried many times. I am from NUS, Singapore. Any helps from you guys?
Thanks,

Hi, I trained the model using your script.
The loss looks like this.

I0815 18:56:21.700711 52129 solver.cpp:631] Iteration 920, lr = 0.001
I0815 18:59:11.323932 52129 solver.cpp:232] Iteration 940, loss = 2.33851
I0815 18:59:11.324184 52129 solver.cpp:631] Iteration 940, lr = 0.001
I0815 19:02:00.977653 52129 solver.cpp:232] Iteration 960, loss = 2.43626
I0815 19:02:00.977931 52129 solver.cpp:631] Iteration 960, lr = 0.001
I0815 19:04:50.639415 52129 solver.cpp:232] Iteration 980, loss = 2.29515
I0815 19:04:50.639688 52129 solver.cpp:631] Iteration 980, lr = 0.001
I0815 19:07:32.759773 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_1000.caffemodel
I0815 19:07:36.681205 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_1000.solverstate
I0815 19:07:40.538120 52129 solver.cpp:432] Iteration 1000, Testing net (#0)
I0815 19:07:58.990095 52129 solver.cpp:488]     Test net output #0: accuracy = 0.685882
I0815 19:07:58.990149 52129 solver.cpp:488]     Test net output #1: loss = 1.75962 (* 1 = 1.75962 loss)
I0815 19:08:07.426625 52129 solver.cpp:232] Iteration 1000, loss = 4.31525
I0815 19:08:07.426946 52129 solver.cpp:631] Iteration 1000, lr = 0.001
I0815 19:10:57.163290 52129 solver.cpp:232] Iteration 1020, loss = 2.93576
I0815 19:10:57.163581 52129 solver.cpp:631] Iteration 1020, lr = 0.001
I0815 19:13:46.859581 52129 solver.cpp:232] Iteration 1040, loss = 2.40981
I0815 19:13:46.859861 52129 solver.cpp:631] Iteration 1040, lr = 0.001
I0815 19:16:36.546717 52129 solver.cpp:232] Iteration 1060, loss = 2.49907
I0815 19:16:36.546993 52129 solver.cpp:631] Iteration 1060, lr = 0.001
I0815 19:19:26.209902 52129 solver.cpp:232] Iteration 1080, loss = 2.27296
I0815 19:19:26.210175 52129 solver.cpp:631] Iteration 1080, lr = 0.001
I0815 19:22:15.858909 52129 solver.cpp:232] Iteration 1100, loss = 2.36718
I0815 19:22:15.859191 52129 solver.cpp:631] Iteration 1100, lr = 0.001
I0815 19:25:05.558161 52129 solver.cpp:232] Iteration 1120, loss = 2.32493
I0815 19:25:05.558434 52129 solver.cpp:631] Iteration 1120, lr = 0.001
I0815 19:27:55.255880 52129 solver.cpp:232] Iteration 1140, loss = 2.39129
I0815 19:27:55.256165 52129 solver.cpp:631] Iteration 1140, lr = 0.001
I0815 19:30:44.819021 52129 solver.cpp:232] Iteration 1160, loss = 2.36149
I0815 19:30:44.819293 52129 solver.cpp:631] Iteration 1160, lr = 0.001
I0815 19:33:34.412870 52129 solver.cpp:232] Iteration 1180, loss = 2.16151
I0815 19:33:34.413147 52129 solver.cpp:631] Iteration 1180, lr = 0.001
I0815 19:36:24.095429 52129 solver.cpp:232] Iteration 1200, loss = 2.36173
I0815 19:36:24.095723 52129 solver.cpp:631] Iteration 1200, lr = 0.001
I0815 19:39:13.749619 52129 solver.cpp:232] Iteration 1220, loss = 2.45683
I0815 19:39:13.749883 52129 solver.cpp:631] Iteration 1220, lr = 0.001
I0815 19:42:03.514300 52129 solver.cpp:232] Iteration 1240, loss = 2.41362
I0815 19:42:03.514590 52129 solver.cpp:631] Iteration 1240, lr = 0.001
I0815 19:44:53.258905 52129 solver.cpp:232] Iteration 1260, loss = 2.29926
I0815 19:44:53.259179 52129 solver.cpp:631] Iteration 1260, lr = 0.001
I0815 19:47:42.936760 52129 solver.cpp:232] Iteration 1280, loss = 2.29143
I0815 19:47:42.937052 52129 solver.cpp:631] Iteration 1280, lr = 0.001
I0815 19:50:32.691732 52129 solver.cpp:232] Iteration 1300, loss = 2.21673
I0815 19:50:32.692003 52129 solver.cpp:631] Iteration 1300, lr = 0.001
I0815 19:53:22.276563 52129 solver.cpp:232] Iteration 1320, loss = 2.30811
I0815 19:53:22.276844 52129 solver.cpp:631] Iteration 1320, lr = 0.001
I0815 19:56:11.834009 52129 solver.cpp:232] Iteration 1340, loss = 2.33596
I0815 19:56:11.834276 52129 solver.cpp:631] Iteration 1340, lr = 0.001
I0815 19:59:01.388342 52129 solver.cpp:232] Iteration 1360, loss = 2.1507
I0815 19:59:01.388628 52129 solver.cpp:631] Iteration 1360, lr = 0.001
I0815 20:01:50.989980 52129 solver.cpp:232] Iteration 1380, loss = 2.25501
I0815 20:01:50.990263 52129 solver.cpp:631] Iteration 1380, lr = 0.001
I0815 20:04:40.654201 52129 solver.cpp:232] Iteration 1400, loss = 2.25313
I0815 20:04:40.654469 52129 solver.cpp:631] Iteration 1400, lr = 0.001
I0815 20:07:30.328219 52129 solver.cpp:232] Iteration 1420, loss = 2.20313
I0815 20:07:30.328493 52129 solver.cpp:631] Iteration 1420, lr = 0.001
I0815 20:10:19.975951 52129 solver.cpp:232] Iteration 1440, loss = 2.20481
I0815 20:10:19.976227 52129 solver.cpp:631] Iteration 1440, lr = 0.001
I0815 20:13:09.721353 52129 solver.cpp:232] Iteration 1460, loss = 2.29201
I0815 20:13:09.721652 52129 solver.cpp:631] Iteration 1460, lr = 0.001
I0815 20:15:59.386811 52129 solver.cpp:232] Iteration 1480, loss = 2.25177
I0815 20:15:59.387084 52129 solver.cpp:631] Iteration 1480, lr = 0.001
I0815 20:18:49.073662 52129 solver.cpp:232] Iteration 1500, loss = 2.22122
I0815 20:18:49.073942 52129 solver.cpp:631] Iteration 1500, lr = 0.001
I0815 20:21:38.805486 52129 solver.cpp:232] Iteration 1520, loss = 2.19175
I0815 20:21:38.805810 52129 solver.cpp:631] Iteration 1520, lr = 0.001
I0815 20:24:28.513808 52129 solver.cpp:232] Iteration 1540, loss = 2.15029
I0815 20:24:28.514101 52129 solver.cpp:631] Iteration 1540, lr = 0.001
I0815 20:27:18.180346 52129 solver.cpp:232] Iteration 1560, loss = 2.21155
I0815 20:27:18.180622 52129 solver.cpp:631] Iteration 1560, lr = 0.001
I0815 20:30:07.847918 52129 solver.cpp:232] Iteration 1580, loss = 2.17659
I0815 20:30:07.848196 52129 solver.cpp:631] Iteration 1580, lr = 0.001
I0815 20:32:57.541263 52129 solver.cpp:232] Iteration 1600, loss = 2.09776
I0815 20:32:57.541563 52129 solver.cpp:631] Iteration 1600, lr = 0.001
I0815 20:35:47.284319 52129 solver.cpp:232] Iteration 1620, loss = 1.99478
I0815 20:35:47.284600 52129 solver.cpp:631] Iteration 1620, lr = 0.001
I0815 20:38:37.013903 52129 solver.cpp:232] Iteration 1640, loss = 2.15796
I0815 20:38:37.014174 52129 solver.cpp:631] Iteration 1640, lr = 0.001
I0815 20:41:26.810891 52129 solver.cpp:232] Iteration 1660, loss = 2.19062
I0815 20:41:26.811189 52129 solver.cpp:631] Iteration 1660, lr = 0.001
I0815 20:44:16.574542 52129 solver.cpp:232] Iteration 1680, loss = 2.07286
I0815 20:44:16.574836 52129 solver.cpp:631] Iteration 1680, lr = 0.001
I0815 20:47:06.222946 52129 solver.cpp:232] Iteration 1700, loss = 2.07381
I0815 20:47:06.223240 52129 solver.cpp:631] Iteration 1700, lr = 0.001
I0815 20:49:55.951117 52129 solver.cpp:232] Iteration 1720, loss = 2.21451
I0815 20:49:55.951405 52129 solver.cpp:631] Iteration 1720, lr = 0.001
I0815 20:52:45.639834 52129 solver.cpp:232] Iteration 1740, loss = 2.07811
I0815 20:52:45.640107 52129 solver.cpp:631] Iteration 1740, lr = 0.001
I0815 20:55:35.355908 52129 solver.cpp:232] Iteration 1760, loss = 2.14357
I0815 20:55:35.356190 52129 solver.cpp:631] Iteration 1760, lr = 0.001
I0815 20:58:25.062911 52129 solver.cpp:232] Iteration 1780, loss = 2.19717
I0815 20:58:25.063179 52129 solver.cpp:631] Iteration 1780, lr = 0.001
I0815 21:01:14.744174 52129 solver.cpp:232] Iteration 1800, loss = 2.07986
I0815 21:01:14.744451 52129 solver.cpp:631] Iteration 1800, lr = 0.001
I0815 21:04:04.339823 52129 solver.cpp:232] Iteration 1820, loss = 2.11781
I0815 21:04:04.340122 52129 solver.cpp:631] Iteration 1820, lr = 0.001
I0815 21:06:53.922601 52129 solver.cpp:232] Iteration 1840, loss = 2.10838
I0815 21:06:53.922893 52129 solver.cpp:631] Iteration 1840, lr = 0.001
I0815 21:09:43.493850 52129 solver.cpp:232] Iteration 1860, loss = 2.08746
I0815 21:09:43.494134 52129 solver.cpp:631] Iteration 1860, lr = 0.001
I0815 21:12:33.116439 52129 solver.cpp:232] Iteration 1880, loss = 1.96505
I0815 21:12:33.116732 52129 solver.cpp:631] Iteration 1880, lr = 0.001
I0815 21:15:22.650295 52129 solver.cpp:232] Iteration 1900, loss = 2.25801
I0815 21:15:22.650579 52129 solver.cpp:631] Iteration 1900, lr = 0.001
I0815 21:18:12.180665 52129 solver.cpp:232] Iteration 1920, loss = 2.0309
I0815 21:18:12.180948 52129 solver.cpp:631] Iteration 1920, lr = 0.001
I0815 21:21:01.732786 52129 solver.cpp:232] Iteration 1940, loss = 1.99836
I0815 21:21:01.733085 52129 solver.cpp:631] Iteration 1940, lr = 0.001
I0815 21:23:51.258694 52129 solver.cpp:232] Iteration 1960, loss = 2.14637
I0815 21:23:51.258970 52129 solver.cpp:631] Iteration 1960, lr = 0.001
I0815 21:26:40.873399 52129 solver.cpp:232] Iteration 1980, loss = 2.11196
I0815 21:26:40.873687 52129 solver.cpp:631] Iteration 1980, lr = 0.001
I0815 21:29:22.737175 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_2000.caffemodel
I0815 21:29:24.311609 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_2000.solverstate
I0815 21:29:27.109315 52129 solver.cpp:432] Iteration 2000, Testing net (#0)
I0815 21:29:45.471135 52129 solver.cpp:488]     Test net output #0: accuracy = 0.69
I0815 21:29:45.471182 52129 solver.cpp:488]     Test net output #1: loss = 2.12226 (* 1 = 2.12226 loss)
I0815 21:29:53.924059 52129 solver.cpp:232] Iteration 2000, loss = 4.92101
I0815 21:29:53.924412 52129 solver.cpp:631] Iteration 2000, lr = 0.001
I0815 21:32:43.561449 52129 solver.cpp:232] Iteration 2020, loss = 3.19778
I0815 21:32:43.561743 52129 solver.cpp:631] Iteration 2020, lr = 0.001
I0815 21:35:33.365213 52129 solver.cpp:232] Iteration 2040, loss = 2.46677
I0815 21:35:33.365488 52129 solver.cpp:631] Iteration 2040, lr = 0.001
I0815 21:38:23.062963 52129 solver.cpp:232] Iteration 2060, loss = 2.20488
I0815 21:38:23.063457 52129 solver.cpp:631] Iteration 2060, lr = 0.001
I0815 21:41:12.869400 52129 solver.cpp:232] Iteration 2080, loss = 2.15901
I0815 21:41:12.869673 52129 solver.cpp:631] Iteration 2080, lr = 0.001
I0815 21:44:02.653378 52129 solver.cpp:232] Iteration 2100, loss = 2.18091
I0815 21:44:02.653652 52129 solver.cpp:631] Iteration 2100, lr = 0.001
I0815 21:46:52.403762 52129 solver.cpp:232] Iteration 2120, loss = 2.2991
I0815 21:46:52.404038 52129 solver.cpp:631] Iteration 2120, lr = 0.001
I0815 21:49:42.154250 52129 solver.cpp:232] Iteration 2140, loss = 2.05538
I0815 21:49:42.154541 52129 solver.cpp:631] Iteration 2140, lr = 0.001
I0815 21:52:31.944736 52129 solver.cpp:232] Iteration 2160, loss = 2.1253
I0815 21:52:31.945013 52129 solver.cpp:631] Iteration 2160, lr = 0.001
I0815 21:55:21.718902 52129 solver.cpp:232] Iteration 2180, loss = 2.0968
I0815 21:55:21.719180 52129 solver.cpp:631] Iteration 2180, lr = 0.001
I0815 21:58:11.463587 52129 solver.cpp:232] Iteration 2200, loss = 2.07307
I0815 21:58:11.463840 52129 solver.cpp:631] Iteration 2200, lr = 0.001
I0815 22:01:01.200736 52129 solver.cpp:232] Iteration 2220, loss = 2.05853
I0815 22:01:01.201009 52129 solver.cpp:631] Iteration 2220, lr = 0.001
I0815 22:03:50.985465 52129 solver.cpp:232] Iteration 2240, loss = 2.06146
I0815 22:03:50.985741 52129 solver.cpp:631] Iteration 2240, lr = 0.001
I0815 22:06:40.745090 52129 solver.cpp:232] Iteration 2260, loss = 2.02932
I0815 22:06:40.745376 52129 solver.cpp:631] Iteration 2260, lr = 0.001
I0815 22:09:30.424286 52129 solver.cpp:232] Iteration 2280, loss = 2.10739
I0815 22:09:30.424578 52129 solver.cpp:631] Iteration 2280, lr = 0.001
I0815 22:12:20.215556 52129 solver.cpp:232] Iteration 2300, loss = 1.98266
I0815 22:12:20.215840 52129 solver.cpp:631] Iteration 2300, lr = 0.001
I0815 22:15:10.013236 52129 solver.cpp:232] Iteration 2320, loss = 2.05712
I0815 22:15:10.013514 52129 solver.cpp:631] Iteration 2320, lr = 0.001
I0815 22:17:59.798974 52129 solver.cpp:232] Iteration 2340, loss = 2.04119
I0815 22:17:59.799223 52129 solver.cpp:631] Iteration 2340, lr = 0.001
I0815 22:20:49.522065 52129 solver.cpp:232] Iteration 2360, loss = 2.10465
I0815 22:20:49.522343 52129 solver.cpp:631] Iteration 2360, lr = 0.001
I0815 22:23:39.197793 52129 solver.cpp:232] Iteration 2380, loss = 2.04238
I0815 22:23:39.198057 52129 solver.cpp:631] Iteration 2380, lr = 0.001
I0815 22:26:28.971088 52129 solver.cpp:232] Iteration 2400, loss = 2.00524
I0815 22:26:28.971369 52129 solver.cpp:631] Iteration 2400, lr = 0.001

It seems that the two 1000 iterations are the same.

这个版本的caffe可以运行在GPU集群上吗？如果可以的话，对集群有要求吗？谢谢

I'm wondering have you ever tried to train the VGG16 temporal model with single-frame, i.e., length=1 instead of length=10? It seems that initialize the model by pre-trained RGB model from ImageNet does not help if I set length=1 when train the temporal flow model.

[ 91%] Building CXX object tools/CMakeFiles/caffe.bin.dir/caffe.cpp.o
Linking CXX executable caffe
../lib/libcaffe.so: undefined reference to cv::VideoCapture::set(int, double)' ../lib/libcaffe.so: undefined reference tocv::VideoCapture::~VideoCapture()'
../lib/libcaffe.so: undefined reference to cv::VideoCapture::isOpened() const' ../lib/libcaffe.so: undefined reference tocv::VideoCapture::get(int) const'
../lib/libcaffe.so: undefined reference to cv::VideoCapture::read(cv::_OutputArray const&)' ../lib/libcaffe.so: undefined reference tocv::VideoCapture::VideoCapture(cv::String const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/caffe] Error 1
make[1]: *** [tools/CMakeFiles/caffe.bin.dir/all] Error 2
make: *** [all] Error 2

After following the instruction in README, I run make&&make install in bulid. However, i got the info above.
I have add the opencv_videoio in Makefile but it still get wrong. How to fix it? Thanks.

Please note somewhere the details of your parallelization. I guess (sorry if I am blind), in the referred paper as well, no pointer to implementation

I have used
build/install/bin/caffe test --model=models/action_recognition/vgg_16_flow_train_val_fast.prototxt --weights=cuhk_action_recognition_16_split1_flow_iter_11000.caffemodel --iterations 100

Not sure about iteration number which one to use?
Is this been used for the published accuracy for optical_flow net?

Hi, @yjxiong @wanglimin thank you for your kindness!
I want to reproduce the pre-trained model for temporal stream. when I used the matlab interface to load the caffe model parameters from the provided caffemodel files, and just simply save the models using the net.save() function, the new sizes of the re-saved caffemodel files are different from the original ones.
For your vgg_16_action_flow_pretrain.caffemodel, the file size of the one you provided is 537083852, the file size of the reproduced one by me is 538738782, as for caffenet, the original file size is 243862418, the re-saved one is 243862097, as for vggnet, the original file size is 553432081, the re-saved one is 553432454. Though the differences are very slight, I still want to know why.
Hope you bother to tell me, thank you in advance!

I am doubting that this Caffe can compute in only a single machine with Multi-GPU.

Thank you!

@wanglimin

ucf101_rgb_img/WallPushups/v_WallPushups_g21_c06 111 98
ucf101_rgb_img/StillRings/v_StillRings_g18_c02 192 85
ucf101_rgb_img/IceDancing/v_IceDancing_g19_c06 251 43
ucf101_rgb_img/Skijet/v_Skijet_g08_c04 208 81
ucf101_rgb_img/Lunges/v_Lunges_g13_c01 266 51
ucf101_rgb_img/CleanAndJerk/v_CleanAndJerk_g16_c04 105 20
ucf101_rgb_img/RopeClimbing/v_RopeClimbing_g13_c04 88 74
ucf101_rgb_img/BaseballPitch/v_BaseballPitch_g14_c01 103 6
ucf101_rgb_img/CuttingInKitchen/v_CuttingInKitchen_g16_c02 197 24
ucf101_rgb_img/SkateBoarding/v_SkateBoarding_g22_c04 85 79
ucf101_rgb_img/RockClimbingIndoor/v_RockClimbingIndoor_g18_c02 168 73
ucf101_rgb_img/FloorGymnastics/v_FloorGymnastics_g24_c01 174 29
ucf101_rgb_img/LongJump/v_LongJump_g16_c03 134 50
ucf101_rgb_img/BalanceBeam/v_BalanceBeam_g23_c04 106 4
ucf101_rgb_img/BoxingPunchingBag/v_BoxingPunchingBag_g12_c05 231 16
ucf101_rgb_img/Bowling/v_Bowling_g20_c02 133 15
ucf101_rgb_img/ThrowDiscus/v_ThrowDiscus_g13_c03 113 92
ucf101_rgb_img/Mixing/v_Mixing_g19_c03 101 53

标签中数字分别表示什么呢？

Since the dataset is very large, I am having a hard time to download it. Would you like to add the script for converting avi files to optical-flow frames?

Since Caffe uses BGR image format, and cv2.imread from OpenCV by default loads image in BGR format, we should not reorder the channels again?
http://github.com/yjxiong/caffe/blob/action_recog/action_python/VideoSpatialPrediction.py#L62

I have installed caffe previously and I can run all the ipynb examples, however I can not get this to work, as I can not find where is that folder for caffe:

mpirun was unable to launch the specified application as it could not find an executable:

Executable:
Node: SATDLS

while attempting to start process rank 0.

4 total processes failed to start
./train_action_recognition_flow.sh: 5: ./train_action_recognition_flow.sh: cmake_build/install/bin/caffe: not found

Hi,

Thanks for the great work, I am using it to fine tune on another action recognition dataset. But right now the situation is the training loss is going down very slowly, but the test loss remain the same. Specifically, starting from iteration 5000, the training loss is 2+ and the test loss is 2+. Now the training loss is around 1, but test loss is still 2+. Could anyone point out any issue or any suggestions? I am using dropout ratio of 0.5 and step size 10000, learning rate start from 0.001. I think maybe overfitting, but I got more than 50 million frames, which shouldn't overfit so quickly.

I0218 15:34:31.817140 38066 solver.cpp:240] Iteration 17680, loss = 0.913562
I0218 15:34:31.817322 38066 solver.cpp:631] Iteration 17680, lr = 0.0001
I0218 15:37:39.009559 38066 solver.cpp:240] Iteration 17700, loss = 0.984204
I0218 15:37:39.009740 38066 solver.cpp:631] Iteration 17700, lr = 0.0001
I0218 15:40:46.215205 38066 solver.cpp:240] Iteration 17720, loss = 0.88048
I0218 15:40:46.215389 38066 solver.cpp:631] Iteration 17720, lr = 0.0001
I0218 15:43:53.435811 38066 solver.cpp:240] Iteration 17740, loss = 0.950506
I0218 15:43:53.435973 38066 solver.cpp:631] Iteration 17740, lr = 0.0001
I0218 15:47:00.659432 38066 solver.cpp:240] Iteration 17760, loss = 0.954575
I0218 15:47:00.659607 38066 solver.cpp:631] Iteration 17760, lr = 0.0001
I0218 15:50:07.868850 38066 solver.cpp:240] Iteration 17780, loss = 0.931296
I0218 15:50:07.869019 38066 solver.cpp:631] Iteration 17780, lr = 0.0001
I0218 15:53:15.080535 38066 solver.cpp:240] Iteration 17800, loss = 0.978098
I0218 15:53:15.080703 38066 solver.cpp:631] Iteration 17800, lr = 0.0001
I0218 15:56:22.298936 38066 solver.cpp:240] Iteration 17820, loss = 0.919493
I0218 15:56:22.299123 38066 solver.cpp:631] Iteration 17820, lr = 0.0001
I0218 15:59:29.498234 38066 solver.cpp:240] Iteration 17840, loss = 0.845611
I0218 15:59:29.498405 38066 solver.cpp:631] Iteration 17840, lr = 0.0001
I0218 16:02:36.685513 38066 solver.cpp:240] Iteration 17860, loss = 0.95367
I0218 16:02:36.685740 38066 solver.cpp:631] Iteration 17860, lr = 0.0001
I0218 16:05:43.915957 38066 solver.cpp:240] Iteration 17880, loss = 0.83995
I0218 16:05:43.916149 38066 solver.cpp:631] Iteration 17880, lr = 0.0001
I0218 16:08:51.107620 38066 solver.cpp:240] Iteration 17900, loss = 0.891882
I0218 16:08:51.107802 38066 solver.cpp:631] Iteration 17900, lr = 0.0001
I0218 16:11:58.317132 38066 solver.cpp:240] Iteration 17920, loss = 1.18598
I0218 16:11:58.317296 38066 solver.cpp:631] Iteration 17920, lr = 0.0001
I0218 16:15:05.533668 38066 solver.cpp:240] Iteration 17940, loss = 1.07879
I0218 16:15:05.533854 38066 solver.cpp:631] Iteration 17940, lr = 0.0001
I0218 16:18:12.735044 38066 solver.cpp:240] Iteration 17960, loss = 1.05319
I0218 16:18:12.735213 38066 solver.cpp:631] Iteration 17960, lr = 0.0001
I0218 16:21:19.916901 38066 solver.cpp:240] Iteration 17980, loss = 0.985989
I0218 16:21:19.917068 38066 solver.cpp:631] Iteration 17980, lr = 0.0001
I0218 16:24:19.163748 38066 solver.cpp:502] Snapshotting to action_vgg_16_rgb_iter_18000.caffemodel
I0218 16:24:27.496953 38066 solver.cpp:510] Snapshotting solver state to action_vgg_16_rgb_iter_18000.solverstate
I0218 16:24:32.482759 38066 solver.cpp:335] Model Synchronization Communication time 0.27824 second
I0218 16:24:32.482831 38066 solver.cpp:428] Iteration 18000, Testing net (#0)
I0218 16:25:11.642127 38066 solver.cpp:484] Test net output #0: accuracy = 0.471176
I0218 16:25:11.642263 38066 solver.cpp:484] Test net output #1: loss = 2.31355 (* 1 = 2.31355 loss)
I0218 16:25:20.908124 38066 solver.cpp:240] Iteration 18000, loss = 1.11894
I0218 16:25:20.908184 38066 solver.cpp:631] Iteration 18000, lr = 0.0001
I0218 16:28:27.909950 38066 solver.cpp:240] Iteration 18020, loss = 0.86253
I0218 16:28:27.910153 38066 solver.cpp:631] Iteration 18020, lr = 0.0001
I0218 16:31:35.093183 38066 solver.cpp:240] Iteration 18040, loss = 0.868103

PS: I also try dropout ratio 0.9, step size 4000 as in the UCF101 example. Around iteration 10000, both training loss and test loss is around 2.0, and doesn't go down. Is there a way to pass this plateau? Thanks

Hi!
@yjxiong

demo.py 运行得到fused_pred 矩阵，我理解每一项为输入测试数据分类为101种行为的概率，不知道是否正确？我从paper（Towards Good Practices for Very Deep Two-Stream ConvNets）中理解，
fused_pred得到概率作为最后的识别率，但是这只是一个视频的数据的结果，想问下paper中的的结果是怎么得到呢？期待您的回复，非常感谢！

@wanglimin Hey, thank you for your work. When I test your models with the vgg_16_rgb_train_val_fast.prototxt, the accuracies are lower than that of your statement. I wonder if there are differences between train_val_fast and train_val which lead to the different accuracies.
Thank you in advance!

ucf101/ParallelBars/v_ParallelBars_g12_c03 165 56
the 165 is the total frame size, what is the meaning of 56?

@wanglimin
when I run "demo.py" I got :

Traceback (most recent call last):
File "demo.py", line 12, in
from VideoSpatialPrediction import VideoSpatialPrediction
File "/home/yuanzhi/tools/caffe/action_python/VideoSpatialPrediction.py", line 11, in
import cv2
ImportError: No module named cv2