araujoalexandre / youtube8m-circulant

Training compact deep learning models for video classification using circulant matrices

Training compact deep learning models for video classification using circulant matrices

This repository contains the code used for the experiments presented in the paper

The repository is a fork of google's repository and borrows from Miech et al

The experiments was run with TensorFlow 1.8 on POWER8 machines with 4 TESLA v100 GPU.

The code is released under Apache License Version 2.0.

Setup

export TRAIN_FOLDER=data/yt8m/frame/ 
export OUT_FOLDER=output_folder 
export N_GPUS=4 

/!\ The files in the output_folder will be erase if you re-run an experiment in the same folder.

Record the global parameters:

export PARAMS="--train_data_pattern=${TRAIN_FOLDER}train*.tfrecord,${TRAIN_FOLDER}validate[A-Za-z]*.tfrecord --frame_features=True --feature_names=rgb,audio --feature_sizes=1024,128 --iterations=15 --sample_random_frames=True --num_gpu=${N_GPUS} --num_readers=100 --batch_size=80 --num_epochs=15 --base_learning_rate=0.0002 --learning_rate_decay=0.8 --data_augmentation=True --n_bagging=10 --moe_num_mixtures=4 --moe_add_batch_norm=True --k_factor=1 --dbof_cluster_size=8192 --fc_hidden_size=512 --fv_cluster_size=64 --netvlad_cluster_size=64 --model=CirulantDiagonalNetwork --start_new_model" 

Experiment 1

python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --video_level_classifier_model=MoeModel ${PARAMS} 
python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --use_d_matrix=True --fc_dbof_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 

Experiment 2

DBoF figure

python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --video_level_classifier_model=MoeModel ${PARAMS}
python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --fc_dbof_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 

NetVLAD figure

python3 code/train.py --train_dir=${OUT_FOLDER} --add_netvlad=True --video_level_classifier_model=MoeModel ${PARAMS} 
python3 code/train.py --train_dir=${OUT_FOLDER} --add_netvlad=True --fc_netvlad_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 

NetFisher figure

python3 code/train.py --train_dir=${OUT_FOLDER} --add_fisher_vector=True --video_level_classifier_model=MoeModel ${PARAMS} 
python3 code/train.py --train_dir=${OUT_FOLDER} --add_fisher_vector=True --fc_netvlad_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 

Experiment 3

python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --no_audio=True --video_level_classifier_model=MoeModel ${PARAMS}
python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --no_audio=True --dbof_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 
python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --no_audio=True --fc_dbof_circulant=True --video_level_classifier_model=MoeModel ${PARAMS} 
python3 code/train.py --train_dir=${OUT_FOLDER} --add_dbof=True --no_audio=True --video_level_classifier_model=Circulant_MoeModel ${PARAMS}