naviocean / pseudo-3d-pytorch

Pseudo-3D Residual Networks

This repo implements the network structure of P3D[1] with PyTorch, pre-trained model weights are converted from caffemodel, which is supported from the author's repo

Requirements:

• pytorch
• numpy
• ffmpeg (for extract image frames from videos)

Pretrained weights

1, P3D-199 trained on Kinetics dataset:

Google Drive url

2, P3D-199 trianed on Kinetics Optical Flow (TVL1):

Google Drive url

Prepare Dataset UCF101

First, download the dataset from UCF into the data folder and then extract it.

cd data && wget http://crcv.ucf.edu/data/UCF101/UCF101.rar
unrar e UCF101.rar

Next, make 3 folders train, test and validation:

mkdir train test validation

Finally, run scripts to extract image frames from videos;

python move.py
python makeVideoFolder.py
python extract.py

Run Code

1, For Training from scratch

python main.py /path/data/

2, For Fine-tuning

python main.py /path/data/ --pretrained

3, For Evaluate model

python main.py /path/data/ --resume=checkpoint.pth.tar --evaluate

4, For testing model

python main.py /path/data/ --test

Experiment Result From Us

Dataset	Accuracy
UCF-101	81.6%
MERL Shopping	82.6%

Reference:

[1]Learning Spatio-Temporal Representation with Pseudo-3D Residual,ICCV2017

pseudo-3d-pytorch's Issues

About　train

Excuse me, when I have prepared UCF101, the problem is produce. Looking forward your reply. Thanks in advance.
python main.py /home/feng/pseudo-3d-pytorch/data
UCF101 data set
Loading P3D model
=> creating model P3D
Traceback (most recent call last):
File "main.py", line 52, in
main()
File "main.py", line 48, in main
Training(name_list=name_list, num_classes=num_classes, modality='RGB', **args)
File "/home/feng/pseudo-3d-pytorch/train.py", line 43, in init
self.train_loader, self.val_loader = self.loading_data()
File "/home/feng/pseudo-3d-pytorch/train.py", line 206, in loading_data
pin_memory=True)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 145, in init
sampler = RandomSampler(dataset)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/sampler.py", line 61, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

About I3D pretrained model.

Sorry to disturb you!
In train.py:

            if self.pretrained:
                self.model = I3D(num_classes=400, modality='rgb')
                self.model.load_state_dict(
                    torch.load('kinetics_i3d_model_rgb.pth'))

Could you tell me where I can get the I3D pretrained moel "kinetics_i3d_model_rgb.pth"?
thank you very much!

MERL Background Class

Thanks for the great work :D, Just want to ask a simple question.
Why did you ignore background class in MERL dataset ??

how to get i3d pretrained model

hi, @naviocean , could you provide kinetics_i3d_model_rgb.pth you used in the i3d code? Thanks very much.

About UCF accuracy

Excuse me. I would like to ask if the accuracy 81.6% you gave on UCF101 dataset is the top1 or the top5?
Since I only have one piece of 1070Ti, my batchsize can only be set to 6.
I trained from scratch according to the steps you gave, and found that the loss cannot converge.
On the test dataset, the accuracy of top1 is 42% and that of top5 is 68%
Do you think the batchsize might have influenced the final result? Thanks for your help.