tteepe / gaitgraph

Official repository for "GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition" (ICIP'21)

GaitGraph

Check out our new version GaitGraph2!

With support for OUMVLP-Pose Dataset, In-Memory Datasets & PyTorch Lightning

Towards a Deeper Understanding of Skeleton-based Gait Recognition

This repository contains the PyTorch code for:

GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition

Torben Teepe, Ali Khan, Johannes Gilg, Fabian Herzog, Stefan Hörmann

Quick Start

Prerequisites

• Python >= 3.6
• CUDA >= 10

First, create a virtual environment or install dependencies directly with:

pip3 install -r requirements.txt

Data preparation

The extraction of the pose data from CASIA-B can either run the commands bellow or download the preprocessed data using:

cd data
sh ./download_data.sh

Optional: If you choose to run the preprocessing, download the dataset and run the following commands.

# Download required weights
cd models
sh ./download_weights.sh

# Copy extraction script
# <PATH_TO_CASIA-B> should be something like: /home/ ... /datasets/CASIA_Gait_Dataset/DatasetB
cd ../data
cp extract_frames.sh <PATH_TO_CASIA-B>

cd <PATH_TO_CASIA-B>
mkdir frames
sh extract_frames.sh
cd frames
find . -type f -regex ".*\.jpg" -print | sort | grep -v bkgrd > ../casia-b_all_frames.csv
cp ../casia-b_all_frames.csv <PATH_TO_REPO>/data

cd <PATH_TO_REPO>/src
export PYTHONPATH=${PWD}:$PYTHONPATH

cd preparation
python3 prepare_detection.py <PATH_TO_CASIA-B> ../../data/casia-b_all_frames.csv ../../data/casia-b_detections.csv
python3 prepare_pose_estimation.py  <PATH_TO_CASIA-B> ../../data/casia-b_detections.csv ../../data/casia-b_pose_coco.csv
python3 split_casia-b.py ../../data/casia-b_pose_coco.csv --output_dir ../../data

Train

To train the model you can run the train.py script. To see all options run:

cd src
export PYTHONPATH=${PWD}:$PYTHONPATH

python3 train.py --help

Check experiments/1_train_*.sh to see the configurations used in the paper.

Optionally start the tensorboard with:

tensorboard --logdir=save/casia-b_tensorboard 

Evaluation

Evaluate the models using evaluate.py script. To see all options run:

python3 evaluate.py --help

Main Results

Top-1 Accuracy per probe angle excluding identical-view cases for the provided models on CASIA-B dataset.

	0	18	36	54	72	90	108	126	144	162	180	mean
NM#5-6	85.3	88.5	91	92.5	87.2	86.5	88.4	89.2	87.9	85.9	81.9	87.7
BG#1-2	75.8	76.7	75.9	76.1	71.4	73.9	78	74.7	75.4	75.4	69.2	74.8
CL#1-2	69.6	66.1	68.8	67.2	64.5	62	69.5	65.6	65.7	66.1	64.3	66.3

The pre-trained model is available here.

Licence & Acknowledgement

GaitPose itself is released under the MIT License (see LICENSE).

The following parts of the code are borrowed from other projects. Thanks for their wonderful work!

• Object Detector: eriklindernoren/PyTorch-YOLOv3
• Pose Estimator: HRNet/HRNet-Human-Pose-Estimation
• ST-GCN Model: yysijie/st-gcn
• ResGCNv1 Model: yfsong0709/ResGCNv1
• SupCon Loss: HobbitLong/SupContrast

Citing GaitGraph

If you use GaitGraph, please use the following BibTeX entry.

@inproceedings{teepe2021gaitgraph,
  author={Teepe, Torben and Khan, Ali and Gilg, Johannes and Herzog, Fabian and H\"ormann, Stefan and Rigoll, Gerhard},
  booktitle={2021 IEEE International Conference on Image Processing (ICIP)}, 
  title={Gait{G}raph: Graph Convolutional Network for Skeleton-Based Gait Recognition}, 
  year={2021},
  pages={2314-2318},
  doi={10.1109/ICIP42928.2021.9506717}
}

gaitgraph's Issues

Requirements Error

Hi, Thanks for the wonderful implementation. I am getting the following error while trying to reproduce Gaitgraph. When I run requirements "!pip3 install -r requirements.txt": I am getting the below error:

ERROR: torchtext 0.9.1 has requirement torch==1.8.1, but you'll have torch 1.7.1 which is incompatible.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.

Kindly let me know if this will affect the end result and if yes how to overcome ?

Thanks
Mritula

Can't reproduce the accuracy in the paper

Hi @tteepe
I run 1_train_casia-b.sh and called ckpt_epoch_best.pth. But the accuracy is a bit low.

I wonder how to reproduce the results in the article. Do I need to set the random seed or reduce lr?
Here is the accuracy graph from tensorboard.

# parameters of the model

When you run train.py, print"# parameters: "

Questions about the training.

As written in the './experiments/1_train_casia-b.sh', the 'train_data_path' and the 'valid_data_path' are set to '../data/casia-b_pose_train_valid.csv' and '../data/casia-b_pose_test.csv', respectively. It seems that the division of the training set (74 subjects) conducted in the data processing, which separates the training set (74) into a training set (59) and a validation set (15), is not used during training. Instead, the testing set (50 subjects) is employed as the validation set to pick the best model.

No offense, but this sounds not reasonable and seems unfair to compare with other skeleton-based gait recognition methods. Under the settings of the training set (59), the validation set (15), and the testing set (50)，the results obtained by me are lower than those written in the paper: NM84.02, CL69.99, BG60.08.

Model doesnt resume training

Hey,

I am trying to train the model as per your paper i.e., 300 epochs with 1e-2 lr, and then for 100 epochs with lr 1e-5. but the model dos not resume training even when the checkpoint_path argument is passed. it just runs for one epoch and shows 0.00 accuracy, and ends training. Any suggestion as to how I can resolve said issue?

Regards,

Yasir

Weights and input information

Hello, I would like to ask whether the input only includes joint information or also joint information.Whether the weights are shared

Angle code is probably wrong

GaitGraph/src/datasets/augmentation.py

Line 43 in 030163b

data_new[2, C, :, :] = np.arccos(data_new[2, i, :, :] / bone_length)

you use a loop, but the data_new[2, C, :, :] just gets the last value after the loop.
According to the distribution of the Multiple Input Branches in the paper('ResGCN'), it's allowed to calculate i th angle of bone by i th bone length

Unable to execute evaluate.py script

Hi, thanks for your work on this project. I'm a newbie to the deep learning environment and currently my research is on Gait Recognition using GCN.

So, the problem I was facing is, after the successful execution of the train.py, what parameters need to be passed to execute the evaluate.py script?

!python3 evaluate.py casia-b -- /content/GaitGraph/data/casia-b_pose_coco.csv \ -- weights_path /content/GaitGraph/models/gaitgraph_resgcn-n39-r8_coco_seq_60.pth \ -- data_path /content/GaitGraph/data/casia-b_pose_valid.csv \

I'm getting this error,

usage: evaluate.py [-h] [--network_name NETWORK_NAME] [--sequence_length SEQUENCE_LENGTH] [--batch_size BATCH_SIZE] [--embedding_layer_size EMBEDDING_LAYER_SIZE] [--use_multi_branch] [--shuffle] {casia-b} weights_path data_path evaluate.py: error: unrecognized arguments: weights_path /content/GaitGraph/models -- data_path

Can you please help me on this issue?

Thank you.

Does 2_evaluate.sh work properly?

Hi, thanks for your work.

I executed Gaitgraph / experiments / 2_evaluate.sh, then it gave the following output.
Is this working properly?

../data/casia-b_pose_valid.csv: 148464: ./074-nm-06-180/000105.jpg,163.31964,59.367306000000006,0.62753356,160.98886000000002,57.036533,0.6496187,163.31964,57.61923,0.6171515999999999,160.40617,57.61923,0.91751564,167.98117,57.61923,0.9350721,156.91002,68.690384,0.90779334,172.64272,68.690384,0.9123460999999999,153.41386,82.09231,0.87953293,174.97348,80.926926,0.877726,153.41386,93.16346999999999,0.86672676,174.97348,90.832695,0.9115324,158.6581,91.998085,0.8330506999999999,169.14656000000002,91.998085,0.8313613999999999,160.40617,110.64424,0.87980276,168.56387,110.0: not found4657,160.40617,129.2904,0.85843027,168.56387,125.79423999999999,0.86784875

A small mistake found in your paper

The results of PoseGait in Table 2 and Table 3 are referenced incorrectly. These results are under the setting of MT(medium-sample training), i.e., 62 for training and the rest for testing. You should use the results of Table 9 in PoseGait for a fair comparison.
By the way, would you like to tell me about the status of your paper? Has it been accepted? I want to cite your good work and only found the arXiv version.

Are you willing to share with us the performance of the GaitGraph on other gait datasets?

Only the performance of the CASIA-B gait dataset is put up on the arxiv version of your paper. In the ./src/common.py, the choices for '--dataset' are ["casia-b", "outdoor-gait", "tum-gaid"]. Would you like to share with us the performance of the GaitGraph on both the "outdoor-gait" and the "tum-gaid"?
In consideration of the relative small number of subjects in "outdoor-gait" and "tum-gaid", have you ever think about the OUMVLP-Pose gait datasets? Both the AlphaPose and the Openpose results are provided in OUMVLP-Pose.
Thanks for the great code sharing and looking forward to your reply.

yolov3 bug

when I run prepare_detection.py, there is a bug。 the weights of yolov3.weights is wrong？

Traceback (most recent call last):
File "prepare_detection.py", line 81, in
detection(**vars(args))
File "prepare_detection.py", line 28, in detection
weights_path="../../models/yolov3.weights",
File "/mnt/cfs/algorithm/users/xianda.guo/code/GaitGraph/src/detector/detector_yolov3.py", line 33, in init
self.model.load_darknet_weights(self.weights_path)
File "/mnt/cfs/algorithm/users/xianda.guo/code/GaitGraph/src/detector/models.py", line 308, in load_darknet_weights
conv_w = torch.from_numpy(weights[ptr : ptr + num_w]).view_as(conv_layer.weight)
RuntimeError: shape '[128, 256, 1, 1]' is invalid for input of size 9038

train bug,gpu bug

when I trained the model,I use 8 gpus [0,1,2,3,4,5,6,7] . But only gpu0 is utilization,the utilization rate of other gpu is 0. I am looking forward your reply,thank you very much!

Download weights error

Hi,
When I try to download weights before train.py, I am getting an error as below saying no such file: How am I to overcome this error?

Thanks
Mritula

!bash ./download_weights.sh
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  236M  100  236M    0     0   104M      0  0:00:02  0:00:02 --:--:--  104M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 33.7M  100 33.7M    0     0  87.5M      0 --:--:-- --:--:-- --:--:-- 87.5M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   647  100   647    0     0   4121      0 --:--:-- --:--:-- --:--:--  4121
100 4023k  100 4023k    0     0  6627k      0 --:--:-- --:--:-- --:--:-- 26.1M
curl: Saved to filename 'gaitgraph_resgcn-n39-r8_coco_seq_60.pth'
Warning: unknown mime-type for "#############################################################" -- using "application/octet-stream"
Error: no such file "#############################################################"
Warning: unknown mime-type for "######## Weights for HRNet Pose Estimation need to ##########" -- using "application/octet-stream"
Error: no such file "######## Weights for HRNet Pose Estimation need to ##########"
Warning: unknown mime-type for "######## be downloaded manually from here:         ##########" -- using "application/octet-stream"
Error: no such file "######## be downloaded manually from here:         ##########"
Warning: unknown mime-type for "######## https://drive.google.com/drive/folders/1nzM_OBV9LbAEA7HClC0chEyf_7ECDXYA" -- using "application/octet-stream"
Error: no such file "######## https://drive.google.com/drive/folders/1nzM_OBV9LbAEA7HClC0chEyf_7ECDXYA"
Warning: unknown mime-type for "######## Files: pose_hrnet_*.pth                   ##########" -- using "application/octet-stream"
Error: no such file "######## Files: pose_hrnet_*.pth                   ##########"
Warning: unknown mime-type for "#############################################################" -- using "application/octet-stream"
Error: no such file "#############################################################"

How to capture training accuracy and validation loss?

Hello @author, thank you for your work and I very much appreciate.

Here is my question, so after training, I understood that only validation accuracy and training loss are captured and are visible clearly when I run with tensor board but to display training accuracy and validation loss, can you please tell me how to achieve that? How can I capture training accuracy and validation loss?

Thank you in advance.

#Question of input

Why don't you use 3d pose for input? Is it less accurate to use 3d pose?

issues of output

Hello author, thank you for your work.
I wonder what these outputs mean
eg:Invalid pose data for: (23, 0, 4, 126) , frame: 18
Invalid pose data for: (25, 0, 4, 36) , frame: 80
Invalid pose data for: (50, 2, 1, 162) , frame: 24
Invalid pose data for: (62, 0, 1, 54) , frame: 3
Invalid pose data for: (69, 0, 5, 162) , frame: 100
Invalid pose data for: (74, 0, 5, 72) , frame: 27
And I also want to know which statements cause these outputs.
thanks!

Please help me! When I run "./1_train_casia-b_fine.sh", I come across an error"ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory".

Questions about the data.

Hello author, thank you for your work. I would like to know how all the video frames recorded in the casia-b_all_frames.csv file are obtained? Because I observed that it may not only save each frame from the video. For example, the video file 015-cl-02-108.avi has 135 frames, but only 65 frames are recorded in your csv file.

Making an Inference Script

I'm attempting to adapt evaluate.py into my own infer.py. Currently I'm just trying to flush a dummy tensor through but I'm getting an error. Any help would be appreciated!!!

Here's my infer.py:

import numpy as np
import torch

from common import get_model_resgcn
from datasets.graph import Graph

# define configuration options with custom class instead of using CLI & argument parser
class opt():
    weights_path = "/app/Gait/GaitGraph/src/models/gaitgraph_resgcn-n39-r8_coco_seq_60.pth"
    network_name = "resgcn-n39-r8"
    embedding_layer_size = 128
    temporal_kernel_size = 9
    dropout = 0.4
    use_multi_branch = True

# Config for dataset
graph = Graph("coco")

# Init model
model, model_args = get_model_resgcn(graph, opt)

if torch.cuda.is_available():
    model.cuda()

# Load weights
checkpoint = torch.load(opt.weights_path)
model.load_state_dict(checkpoint["model"])

model.eval()

# Load data
data_arr = np.ones((51,60,1))
data_ten = torch.from_numpy(data_arr)

# Calculate embeddings
with torch.no_grad():
    bsz = data_ten.shape[0]
    data_ten_flipped = torch.flip(data_ten, dims=[1])
    data_ten = torch.cat([data_ten, data_ten_flipped], dim=0)

    if torch.cuda.is_available():
        data_ten = data_ten.cuda(non_blocking=True)

    output = model(data_ten)
    f1, f2 = torch.split(output, [bsz, bsz], dim=0)
    output = torch.mean(torch.stack([f1, f2]), dim=0)

Here's the error message I get:

Traceback (most recent call last):
  File "/app/Gait/GaitGraph/src/infer.py", line 27, in <module>
    model.load_state_dict(checkpoint["model"])
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ResGCN:
        Missing key(s) in state_dict: "input_branches.1.A", "input_branches.1.bn.weight", "input_branches.1.bn.bias", "input_branches.1.bn.running_mean", "input_branches.1.bn.running_var", "input_branches.1.layers.0.edge", "input_branches.1.layers.0.scn.conv.gcn.weight", "input_branches.1.layers.0.scn.conv.gcn.bias", "input_branches.1.layers.0.scn.bn.weight", "input_branches.1.layers.0.scn.bn.bias", "input_branches.1.layers.0.scn.bn.running_mean", "input_branches.1.layers.0.scn.bn.running_var", "input_branches.1.layers.0.tcn.conv.weight", "input_branches.1.layers.0.tcn.conv.bias", "input_branches.1.layers.0.tcn.bn.weight", "input_branches.1.layers.0.tcn.bn.bias", "input_branches.1.layers.0.tcn.bn.running_mean", "input_branches.1.layers.0.tcn.bn.running_var", "input_branches.1.layers.1.edge", "input_branches.1.layers.1.scn.conv_down.weight", "input_branches.1.layers.1.scn.conv_down.bias", "input_branches.1.layers.1.scn.bn_down.weight", "input_branches.1.layers.1.scn.bn_down.bias", "input_branches.1.layers.1.scn.bn_down.running_mean", "input_branches.1.layers.1.scn.bn_down.running_var", "input_branches.1.layers.1.scn.conv.gcn.weight", "input_branches.1.layers.1.scn.conv.gcn.bias", "input_branches.1.layers.1.scn.bn.weight", "input_branches.1.layers.1.scn.bn.bias", "input_branches.1.layers.1.scn.bn.running_mean", "input_branches.1.layers.1.scn.bn.running_var", "input_branches.1.layers.1.scn.conv_up.weight", "input_branches.1.layers.1.scn.conv_up.bias", "input_branches.1.layers.1.scn.bn_up.weight", "input_branches.1.layers.1.scn.bn_up.bias", "input_branches.1.layers.1.scn.bn_up.running_mean", "input_branches.1.layers.1.scn.bn_up.running_var", "input_branches.1.layers.1.tcn.conv_down.weight", "input_branches.1.layers.1.tcn.conv_down.bias", "input_branches.1.layers.1.tcn.bn_down.weight", "input_branches.1.layers.1.tcn.bn_down.bias", "input_branches.1.layers.1.tcn.bn_down.running_mean", "input_branches.1.layers.1.tcn.bn_down.running_var", "input_branches.1.layers.1.tcn.conv.weight", "input_branches.1.layers.1.tcn.conv.bias", "input_branches.1.layers.1.tcn.bn.weight", "input_branches.1.layers.1.tcn.bn.bias", "input_branches.1.layers.1.tcn.bn.running_mean", "input_branches.1.layers.1.tcn.bn.running_var", "input_branches.1.layers.1.tcn.conv_up.weight", "input_branches.1.layers.1.tcn.conv_up.bias", "input_branches.1.layers.1.tcn.bn_up.weight", "input_branches.1.layers.1.tcn.bn_up.bias", "input_branches.1.layers.1.tcn.bn_up.running_mean", "input_branches.1.layers.1.tcn.bn_up.running_var", "input_branches.1.layers.2.edge", "input_branches.1.layers.2.scn.residual.0.weight", "input_branches.1.layers.2.scn.residual.0.bias", "input_branches.1.layers.2.scn.residual.1.weight", "input_branches.1.layers.2.scn.residual.1.bias", "input_branches.1.layers.2.scn.residual.1.running_mean", "input_branches.1.layers.2.scn.residual.1.running_var", "input_branches.1.layers.2.scn.conv_down.weight", "input_branches.1.layers.2.scn.conv_down.bias", "input_branches.1.layers.2.scn.bn_down.weight", "input_branches.1.layers.2.scn.bn_down.bias", "input_branches.1.layers.2.scn.bn_down.running_mean", "input_branches.1.layers.2.scn.bn_down.running_var", "input_branches.1.layers.2.scn.conv.gcn.weight", "input_branches.1.layers.2.scn.conv.gcn.bias", "input_branches.1.layers.2.scn.bn.weight", "input_branches.1.layers.2.scn.bn.bias", "input_branches.1.layers.2.scn.bn.running_mean", "input_branches.1.layers.2.scn.bn.running_var", "input_branches.1.layers.2.scn.conv_up.weight", "input_branches.1.layers.2.scn.conv_up.bias", "input_branches.1.layers.2.scn.bn_up.weight", "input_branches.1.layers.2.scn.bn_up.bias", "input_branches.1.layers.2.scn.bn_up.running_mean", "input_branches.1.layers.2.scn.bn_up.running_var", "input_branches.1.layers.2.tcn.conv_down.weight", "input_branches.1.layers.2.tcn.conv_down.bias", "input_branches.1.layers.2.tcn.bn_down.weight", "input_branches.1.layers.2.tcn.bn_down.bias", "input_branches.1.layers.2.tcn.bn_down.running_mean", "input_branches.1.layers.2.tcn.bn_down.running_var", "input_branches.1.layers.2.tcn.conv.weight", "input_branches.1.layers.2.tcn.conv.bias", "input_branches.1.layers.2.tcn.bn.weight", "input_branches.1.layers.2.tcn.bn.bias", "input_branches.1.layers.2.tcn.bn.running_mean", "input_branches.1.layers.2.tcn.bn.running_var", "input_branches.1.layers.2.tcn.conv_up.weight", "input_branches.1.layers.2.tcn.conv_up.bias", "input_branches.1.layers.2.tcn.bn_up.weight", "input_branches.1.layers.2.tcn.bn_up.bias", "input_branches.1.layers.2.tcn.bn_up.running_mean", "input_branches.1.layers.2.tcn.bn_up.running_var", "input_branches.2.A", "input_branches.2.bn.weight", "input_branches.2.bn.bias", "input_branches.2.bn.running_mean", "input_branches.2.bn.running_var", "input_branches.2.layers.0.edge", "input_branches.2.layers.0.scn.conv.gcn.weight", "input_branches.2.layers.0.scn.conv.gcn.bias", "input_branches.2.layers.0.scn.bn.weight", "input_branches.2.layers.0.scn.bn.bias", "input_branches.2.layers.0.scn.bn.running_mean", "input_branches.2.layers.0.scn.bn.running_var", "input_branches.2.layers.0.tcn.conv.weight", "input_branches.2.layers.0.tcn.conv.bias", "input_branches.2.layers.0.tcn.bn.weight", "input_branches.2.layers.0.tcn.bn.bias", "input_branches.2.layers.0.tcn.bn.running_mean", "input_branches.2.layers.0.tcn.bn.running_var", "input_branches.2.layers.1.edge", "input_branches.2.layers.1.scn.conv_down.weight", "input_branches.2.layers.1.scn.conv_down.bias", "input_branches.2.layers.1.scn.bn_down.weight", "input_branches.2.layers.1.scn.bn_down.bias", "input_branches.2.layers.1.scn.bn_down.running_mean", "input_branches.2.layers.1.scn.bn_down.running_var", "input_branches.2.layers.1.scn.conv.gcn.weight", "input_branches.2.layers.1.scn.conv.gcn.bias", "input_branches.2.layers.1.scn.bn.weight", "input_branches.2.layers.1.scn.bn.bias", "input_branches.2.layers.1.scn.bn.running_mean", "input_branches.2.layers.1.scn.bn.running_var", "input_branches.2.layers.1.scn.conv_up.weight", "input_branches.2.layers.1.scn.conv_up.bias", "input_branches.2.layers.1.scn.bn_up.weight", "input_branches.2.layers.1.scn.bn_up.bias", "input_branches.2.layers.1.scn.bn_up.running_mean", "input_branches.2.layers.1.scn.bn_up.running_var", "input_branches.2.layers.1.tcn.conv_down.weight", "input_branches.2.layers.1.tcn.conv_down.bias", "input_branches.2.layers.1.tcn.bn_down.weight", "input_branches.2.layers.1.tcn.bn_down.bias", "input_branches.2.layers.1.tcn.bn_down.running_mean", "input_branches.2.layers.1.tcn.bn_down.running_var", "input_branches.2.layers.1.tcn.conv.weight", "input_branches.2.layers.1.tcn.conv.bias", "input_branches.2.layers.1.tcn.bn.weight", "input_branches.2.layers.1.tcn.bn.bias", "input_branches.2.layers.1.tcn.bn.running_mean", "input_branches.2.layers.1.tcn.bn.running_var", "input_branches.2.layers.1.tcn.conv_up.weight", "input_branches.2.layers.1.tcn.conv_up.bias", "input_branches.2.layers.1.tcn.bn_up.weight", "input_branches.2.layers.1.tcn.bn_up.bias", "input_branches.2.layers.1.tcn.bn_up.running_mean", "input_branches.2.layers.1.tcn.bn_up.running_var", "input_branches.2.layers.2.edge", "input_branches.2.layers.2.scn.residual.0.weight", "input_branches.2.layers.2.scn.residual.0.bias", "input_branches.2.layers.2.scn.residual.1.weight", "input_branches.2.layers.2.scn.residual.1.bias", "input_branches.2.layers.2.scn.residual.1.running_mean", "input_branches.2.layers.2.scn.residual.1.running_var", "input_branches.2.layers.2.scn.conv_down.weight", "input_branches.2.layers.2.scn.conv_down.bias", "input_branches.2.layers.2.scn.bn_down.weight", "input_branches.2.layers.2.scn.bn_down.bias", "input_branches.2.layers.2.scn.bn_down.running_mean", "input_branches.2.layers.2.scn.bn_down.running_var", "input_branches.2.layers.2.scn.conv.gcn.weight", "input_branches.2.layers.2.scn.conv.gcn.bias", "input_branches.2.layers.2.scn.bn.weight", "input_branches.2.layers.2.scn.bn.bias", "input_branches.2.layers.2.scn.bn.running_mean", "input_branches.2.layers.2.scn.bn.running_var", "input_branches.2.layers.2.scn.conv_up.weight", "input_branches.2.layers.2.scn.conv_up.bias", "input_branches.2.layers.2.scn.bn_up.weight", "input_branches.2.layers.2.scn.bn_up.bias", "input_branches.2.layers.2.scn.bn_up.running_mean", "input_branches.2.layers.2.scn.bn_up.running_var", "input_branches.2.layers.2.tcn.conv_down.weight", "input_branches.2.layers.2.tcn.conv_down.bias", "input_branches.2.layers.2.tcn.bn_down.weight", "input_branches.2.layers.2.tcn.bn_down.bias", "input_branches.2.layers.2.tcn.bn_down.running_mean", "input_branches.2.layers.2.tcn.bn_down.running_var", "input_branches.2.layers.2.tcn.conv.weight", "input_branches.2.layers.2.tcn.conv.bias", "input_branches.2.layers.2.tcn.bn.weight", "input_branches.2.layers.2.tcn.bn.bias", "input_branches.2.layers.2.tcn.bn.running_mean", "input_branches.2.layers.2.tcn.bn.running_var", "input_branches.2.layers.2.tcn.conv_up.weight", "input_branches.2.layers.2.tcn.conv_up.bias", "input_branches.2.layers.2.tcn.bn_up.weight", "input_branches.2.layers.2.tcn.bn_up.bias", "input_branches.2.layers.2.tcn.bn_up.running_mean", "input_branches.2.layers.2.tcn.bn_up.running_var". 
        size mismatch for input_branches.0.bn.weight: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([6]).
        size mismatch for input_branches.0.bn.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([6]).
        size mismatch for input_branches.0.bn.running_mean: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([6]).
        size mismatch for input_branches.0.bn.running_var: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([6]).
        size mismatch for input_branches.0.layers.0.scn.conv.gcn.weight: copying a param with shape torch.Size([192, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([192, 6, 1, 1]).
        size mismatch for main_stream.0.scn.residual.0.weight: copying a param with shape torch.Size([128, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 96, 1, 1]).
        size mismatch for main_stream.0.scn.conv_down.weight: copying a param with shape torch.Size([16, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([16, 96, 1, 1]).

# parameters of the model

When you run train.py, print"# parameters: "

Accuracy of models trained form scratch

Hi @tteepe
I am very interested in this work and have read your paper.
I tried to train the resgcn model from scratch on casia-b dataset (by download_data.sh) following experiments/1_train_casia-b.sh .

As shown blew, the accuracy of best models are lower than gaitgraph_resgcn-n39-r8_coco_seq_60.pth.
The mean acc of CL is 50.8 (51.1 after tune) while that of gaitgraph_resgcn-n39-r8_coco_seq_60.pth is 66.3.
Anything wrong in my experiment settings? Does it need a pretrained weight file from https://github.com/yfsong0709/ResGCNv1?

1. experiments/1_train_casia-b.sh

	0	18	36	54	72	90	108	126	144	162	180	mean
NM#5-6	75.1	77.4	81.4	82.9	78.6	80.5	84	78.5	82.6	77.7	70.8	79
BG#1-2	65.2	67.4	73	67.7	66.3	67.7	65.8	65.2	65	70.3	60.7	66.8
CL#1-2	51	51.8	41.6	49	51.2	49.7	54.9	54.7	52.4	52.6	50	50.8

2. experiments/1_train_casia-b_fine.sh

	0	18	36	54	72	90	108	126	144	162	180	mean
NM#5-6	77.9	80.1	80.9	82.9	78.4	81.5	84.7	77.5	82	78.7	71.6	79.7
BG#1-2	67.5	69.5	73.6	70.1	67	67.8	66.4	65.2	64.4	71.5	61.4	67.7
CL#1-2	52.4	52.4	43.5	48	52.2	51.4	54.8	53	51.8	53.2	49.6	51.1

About Input information.

Hi! Thanks for your great work!
I have a question about the input skeleton data. Hrnet give a joint two dims (x,y)? what is the 3rd dim means?

ResGCN module

Hi,
I am looking up for ResGCN structure. Where can I find it in this repo? I understand from the below statement that it should be in the ,nets folder but I couldnt find .nets. Can you kindly help me on this?

from .nets import ResGCN

Thanks
Mritula

Question of 2D coordinates

May I ask whether the 2d data of an input picture refers to the coordinates in the original picture (Picture 1) or the coordinates of the Bbox (Picture 2) obtained by target detection from the original picture?

---

请问输入的一张图片的二维数据，是指原始图片（图片1）中的坐标，还是从原始图片中进行目标检测得到的bbox（图片2）的坐标呢？