tteepe / gaitgraph2

Official code for "Towards a Deeper Understanding of Skeleton-based Gait Recognition" (CVPRW'22)

GaitGraph2: Towards a Deeper Understanding of Skeleton-based Gait Recognition

Accepted at: 17th IEEE Computer Society Workshop on Biometrics 2022 (CVPRW'22)

Previous version:
GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition (ICIP'21)

Preparation

Clone the repository and install the dependencies from requirements.txt.

Datasets

• CASIA-B: Download from here and move casia-b_pose_coco.csv to data
• OUMVLP-Pose: Download from here

Pre-trained models

You can download pre-trained models from here.

Running the code

We use PyTorch Lightning CLI for configuration and training.

Train:

# CASIA-B
python3 gaitgraph_casia_b.py fit --config configs/casia_b.yaml 
# OUMVLP-Pose (OpenPose)
python3 gaitgraph_oumvlp.py fit --config configs/oumvlp.yaml
# OUMVLP-Pose (AlphaPose)
python3 gaitgraph_oumvlp.py fit --config configs/oumvlp.yaml --data.keypoints alphapose

Test:

python3 gaitgraph_{casia_b,oumvlp}.py predict --config <path_to_config_file> --ckpt_path <path_to_checkpoint> --model.tta True

Logs and checkpoints will be saved to lighting_logs and can be shown in tensorboard with:

tensorboard --logdir lightning_logs

Visualization of Activations

To run the visualization add run the following command:

python3 gaitgraph_{casia_b,oumvlp}.py predict --config <path_to_config_file> --ckpt_path <path_to_checkpoint> --config configs/draw.yaml --model.tta False

Examples:

Citation

If you use GaitGraph2, please use the following BibTeX entry.

@misc{teepe2022gaitgraph2,
  doi = {10.48550/ARXIV.2204.07855},
  url = {https://arxiv.org/abs/2204.07855},
  author = {Teepe, Torben and Gilg, Johannes and Herzog, Fabian and H\"ormann, Stefan and Rigoll, Gerhard},
  title = {Towards a Deeper Understanding of Skeleton-based Gait Recognition},
  publisher = {arXiv},
  year = {2022}
}

gaitgraph2's Issues

RuntimeError: running_mean should contain 5 elements not 15

/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py:896: UserWarning: Failed parsing docstring for <function Trainer.fit at 0x7f164af54dc0>: docstring-parser package is required
rank_zero_warn(f"Failed parsing docstring for {component}: docstring-parser package is required")
/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py:896: UserWarning: Failed parsing docstring for <function Trainer.validate at 0x7f164af55000>: docstring-parser package is required
rank_zero_warn(f"Failed parsing docstring for {component}: docstring-parser package is required")
/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py:896: UserWarning: Failed parsing docstring for <function Trainer.test at 0x7f164af55240>: docstring-parser package is required
rank_zero_warn(f"Failed parsing docstring for {component}: docstring-parser package is required")
/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py:896: UserWarning: Failed parsing docstring for <function Trainer.predict at 0x7f164af55480>: docstring-parser package is required
rank_zero_warn(f"Failed parsing docstring for {component}: docstring-parser package is required")
/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py:896: UserWarning: Failed parsing docstring for <function Trainer.tune at 0x7f164af55630>: docstring-parser package is required
rank_zero_warn(f"Failed parsing docstring for {component}: docstring-parser package is required")
Global seed set to 5318008
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params

0 | backbone | ResGCN | 255 K
1 | distance | LpDistance | 0
2 | train_loss | SupConLoss | 0
3 | val_loss | ContrastiveLoss | 0

255 K Trainable params
0 Non-trainable params
255 K Total params
1.023 Total estimated model params size (MB)
Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/gaitgraph_casia_b.py", line 287, in
cli_main()
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/gaitgraph_casia_b.py", line 277, in cli_main
LightningCLI(
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py", line 566, in init
self._run_subcommand(self.subcommand)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/utilities/cli.py", line 837, in _run_subcommand
fn(**fn_kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 770, in fit
self._call_and_handle_interrupt(
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 723, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 811, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1236, in _run
results = self._run_stage()
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1323, in _run_stage
return self._run_train()
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1345, in _run_train
self._run_sanity_check()
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1413, in _run_sanity_check
val_loop.run()
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 155, in advance
dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 128, in advance
output = self._evaluation_step(**kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 226, in _evaluation_step
output = self.trainer._call_strategy_hook("validation_step", *kwargs.values())
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1765, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 344, in validation_step
return self.model.validation_step(*args, **kwargs)
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/gaitgraph_casia_b.py", line 81, in validation_step
y_hat = self(x)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/gaitgraph_casia_b.py", line 68, in forward
return self.backbone(x)[0]
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/models/ResGCNv1/nets.py", line 65, in forward
x_cat.append(branch(x[:, i]))
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/brianw0924/Desktop/GaitGraph2/GaitGraph/models/ResGCNv1/nets.py", line 24, in forward
x = self.bn(x)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/modules/batchnorm.py", line 168, in forward
return F.batch_norm(
File "/home/brianw0924/anaconda3/envs/GaitGraph2/lib/python3.10/site-packages/torch/nn/functional.py", line 2421, in batch_norm
return torch.batch_norm(
RuntimeError: running_mean should contain 5 elements not 15

I run

python gaitgraph_casia_b.py fit --config configs/casia_b.yaml

and get this

Hi, can you please tell me how to fix this error?

About the Paper of GaitGraph2

Thank you for your work in gait recognition. I have read your last paper GaitGraph. It really inspires me. I can't help but read your latest work in GaitGraph2.But I can not find the paper on the internet.
If possible, can you send me a copy of the paper? Or tell me how can I find it on the internet. I would appreciate it.

RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

I have cuDNN installed but still I am getting this error

File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\gaitgraph_casia_b.py", line 300, in
cli_main()
File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\gaitgraph_casia_b.py", line 290, in cli_main
LightningCLI(
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\cli.py", line 353, in init
self._run_subcommand(self.subcommand)
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\cli.py", line 642, in _run_subcommand
fn(**fn_kwargs)
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 529, in fit
call._call_and_handle_interrupt(
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\call.py", line 42, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 568, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 973, in _run
results = self._run_stage()
^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1014, in _run_stage
self._run_sanity_check()
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1043, in _run_sanity_check
val_loop.run()
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\loops\utilities.py", line 177, in _decorator
return loop_run(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\loops\evaluation_loop.py", line 115, in run
self._evaluation_step(batch, batch_idx, dataloader_idx)
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\loops\evaluation_loop.py", line 375, in _evaluation_step
output = call._call_strategy_hook(trainer, hook_name, *step_kwargs.values())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\trainer\call.py", line 291, in _call_strategy_hook
output = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\strategies\strategy.py", line 379, in validation_step
return self.model.validation_step(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\gaitgraph_casia_b.py", line 92, in validation_step
y_hat = self(x)
^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\gaitgraph_casia_b.py", line 79, in forward
return self.backbone(x)[0]
^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\models\ResGCNv1\nets.py", line 65, in forward
x_cat.append(branch(x[:, i]))
^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\FYP\GaitGraph2-main\GaitGraph2-main\GaitGraph\models\ResGCNv1\nets.py", line 24, in forward
x = self.bn(x)
^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\batchnorm.py", line 171, in forward
return F.batch_norm(
^^^^^^^^^^^^^
File "C:\Users\Pc\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\functional.py", line 2450, in batch_norm
return torch.batch_norm(
^^^^^^^^^^^^^^^^^
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

error when run prepare_detection.py

maybe there is something wrong with pre-trained model?
when I python prepare_detection.py, I got this:
Traceback (most recent call last):
File "/home/cver/GithubExec/GaitGraph/src/preparation/prepare_detection.py", line 78, in
detection(**vars(args))
File "/home/cver/GithubExec/GaitGraph/src/preparation/prepare_detection.py", line 26, in detection
detector = DetectorYOLOv3(
File "/home/cver/GithubExec/GaitGraph/src/detector/detector_yolov3.py", line 33, in init
self.model.load_darknet_weights(self.weights_path)
File "/home/cver/GithubExec/GaitGraph/src/detector/models.py", line 307, in load_darknet_weights
conv_w = torch.from_numpy(weights[ptr : ptr + num_w]).view_as(conv_layer.weight)
RuntimeError: shape '[1024, 512, 3, 3]' is invalid for input of size 4617550

AttributeError: 'tuple' object has no attribute 'get' in test

When I do test, I face this problem of restoring the checkpoint

The command I use is python3 gaitgraph_casia_b.py predict --config configs/casia_b.yaml --ckpt_path /home/fuyang/KeyPoint/GaitGraph2-main/data/processed/casia_b_test.pt --model.tta True

Restoring states from the checkpoint path at /home/fuyang/KeyPoint/GaitGraph2-main/data/processed/casia_b_test.pt
Traceback (most recent call last):
File "gaitgraph_casia_b.py", line 283, in
cli_main()
File "gaitgraph_casia_b.py", line 273, in cli_main
LightningCLI(
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 566, in init
self._run_subcommand(self.subcommand)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 837, in _run_subcommand
fn(**fn_kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1023, in predict
return self._call_and_handle_interrupt(
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 721, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1070, in _predict_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1177, in _run
self._restore_modules_and_callbacks(ckpt_path)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1138, in _restore_modules_and_callbacks
self._checkpoint_connector._restore_quantization_callbacks()
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 230, in _restore_quantization_callbacks
callback_states = self._loaded_checkpoint.get("callbacks")
AttributeError: 'tuple' object has no attribute 'get'

where can I find the checkpoint file ?

Hi, I use the training command to finish the model train in caisa-b , but I can't find the checkpoint --ckpt_path <path_to_checkpoint> of the trained model when I do inference? Can you tell me where can I find the checkpoint file path or how to set the output checkpoint path?
Thank you very much!

Backbone mismatch between CASIA-B pre-trained model and model config.

@tteepe
I faced an issue when trying to run predict on CASIA-B, there were many unexpected layers not recognized when loading state_dict.
The result of my investigation points out that there was a backend mismatch, more details below:

• In casia_b.yaml: backend_name: resgcn-n21-r8.
• However, the pre-trained model used backbone resgcn-n31-r8 as stated in hparams.yaml and the training log file of this model as well.
• There is no such n31 structure defined in the __init__.py of ResGCNv1, thus changing the backend_name solves nothing.

No module named evaluate

When I tried to run predict on casia-b with pre-trained resgcn-n39-r8 model, there was an error showing: No module named evaluate, and the specific sub-module that was required is _evaluate_casia_b.

RuntimeError: number of dims don't match in permute

when I run this command, and change ' multi_branch: bool = False' to ' multi_branch: bool = True'.

python3 gaitgraph_casia_b.py --config configs/casia_b.yaml

It doesn't work, I wonder why.

Global seed set to 5318008
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3,4,5,6,7]

  | Name       | Type            | Params
-----------------------------------------------
0 | backbone   | ResGCN          | 350 K 
1 | distance   | LpDistance      | 0     
2 | train_loss | SupConLoss      | 0     
3 | val_loss   | ContrastiveLoss | 0     
-----------------------------------------------
350 K     Trainable params
0         Non-trainable params
350 K     Total params
1.403     Total estimated model params size (MB)
Global seed set to 5318008                                                                                                                                                                                                                                                                 
Epoch 0:   0%|                                                                                                                                                                                                                                            | 0/11 [00:00<00:00, 2770.35it/s]Traceback (most recent call last):
  File "gaitgraph_casia_b.py", line 295, in <module>
    cli_main()
  File "gaitgraph_casia_b.py", line 289, in cli_main
    save_config_overwrite=True
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/utilities/cli.py", line 289, in __init__
    self.fit()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/utilities/cli.py", line 432, in fit
    self.trainer.fit(**self.fit_kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 552, in fit
    self._run(model)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 922, in _run
    self._dispatch()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 990, in _dispatch
    self.accelerator.start_training(self)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 92, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 161, in start_training
    self._results = trainer.run_stage()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1000, in run_stage
    return self._run_train()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1049, in _run_train
    self.fit_loop.run()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 111, in run
    self.advance(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/fit_loop.py", line 200, in advance
    epoch_output = self.epoch_loop.run(train_dataloader)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 111, in run
    self.advance(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 130, in advance
    batch_output = self.batch_loop.run(batch, self.iteration_count, self._dataloader_idx)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 100, in run
    super().run(batch, batch_idx, dataloader_idx)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 111, in run
    self.advance(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 147, in advance
    result = self._run_optimization(batch_idx, split_batch, opt_idx, optimizer)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 201, in _run_optimization
    self._optimizer_step(optimizer, opt_idx, batch_idx, closure)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 403, in _optimizer_step
    using_lbfgs=is_lbfgs,
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/core/lightning.py", line 1616, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/core/optimizer.py", line 206, in step
    self.__optimizer_step(closure=closure, profiler_name=profiler_name, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/core/optimizer.py", line 128, in __optimizer_step
    trainer.accelerator.optimizer_step(self._optimizer, self._optimizer_idx, lambda_closure=closure, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 296, in optimizer_step
    self.run_optimizer_step(optimizer, opt_idx, lambda_closure, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 303, in run_optimizer_step
    self.training_type_plugin.optimizer_step(optimizer, lambda_closure=lambda_closure, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 226, in optimizer_step
    optimizer.step(closure=lambda_closure, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/optim/lr_scheduler.py", line 65, in wrapper
    return wrapped(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/optim/optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/optim/adamw.py", line 65, in step
    loss = closure()
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 235, in _training_step_and_backward_closure
    result = self.training_step_and_backward(split_batch, batch_idx, opt_idx, optimizer, hiddens)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 536, in training_step_and_backward
    result = self._training_step(split_batch, batch_idx, opt_idx, hiddens)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 306, in _training_step
    training_step_output = self.trainer.accelerator.training_step(step_kwargs)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 193, in training_step
    return self.training_type_plugin.training_step(*step_kwargs.values())
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 172, in training_step
    return self.model.training_step(*args, **kwargs)
  File "gaitgraph_casia_b.py", line 75, in training_step
    y_hat = self(x)
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "gaitgraph_casia_b.py", line 68, in forward
    return self.backbone(x)[0]
  File "/home/jdy/anaconda3/envs/gaitgraph2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/jdy/GaitGraph2/GaitGraph/models/ResGCNv1/nets.py", line 61, in forward
    x = x.permute(0, 3, 4, 1, 2)
RuntimeError: number of dims don't match in permute

I have struggle with the requirements.txt.

I try the commands with pip install -r requirements. And I got so many mistakes, sush as
"No module named 'pytorch_lightning.utilities.cli'".
However, I do find the pytorch-lightning 2.0.2 pypi_0 pypi with "conda list".
So could the author provide the exact version of the pytorch-lightning[extra], torchvision?
I am new with the pytorch-lightning, the install commands is also greatly helpful for me!

Please share the ID_list.csv

Hi @tteepe ,
Thanks for sharing this great job.
I can't find the ID_list.csv in OUMVLP-Pose dataset, could you please share it?

How to know the accuracy of the test under the Alphapose dataset of OUMVLP-Pose

I am training with the AlphaPose dataset from OUMVLP-Pose, but when I look at the training results at the end of the training, I can only see epoch, hp_metric, lr_Adam, train_loss_epoch, train_loss_step, val_loss_epoch, val_loss_step results, but I can't see the test accuracy. However, the accuracy can be seen when using the CASIA-B dataset.

When I checked the code of gaitgraph_oumvlp.py, I found that the train_epoch_end() function has the definition, calculation and logging of the accuracy, but this function is not called, what should I do if I want to call this function?

Thanks in advance for the answer!
:)

About Mparams and GFLOPs of GaitGraph2

Hi, it is a great work! I want to know Mparams and GFLOPs of GaitGraph2. Can you help me? Thank you very much!!

Issues with weights for the model, missing keys and wrong size.

Hello!
I am trying to use the pre-trained model to the inference.
Unfortunately, I am failing to load the provided weights.
I use this model:
backend_name="resgcn-n39-r8" with parameters:
model_args = { "A": torch.tensor(graph.A, dtype=torch.float32, requires_grad=False), "num_class": embedding_layer_size, "num_input": 3 if multi_branch else 1, "num_channel": 5 if multi_input else 3, "parts": graph.parts, }
And weights:
gaitgraph_resgcn-n39-r8_coco_seq_60.pth

However, the keys and sizes between the model and the saved dictionary do not match, so I cannot load from the checkpoint, and many keys are missing:
load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ResGCN: Missing key(s) in state_dict: "input_branches.1.A", "input_branches.1.bn.weight", "input_branches.1.bn.bias", "input_branches.1.bn.running_mean", "input_branches.1.bn.running_var", "input_branches.1.layers.0.edge", "input_branches.1.layers.0.scn.conv.gcn.weight", "input_branches.1.layers.0.scn.conv.gcn.bias", "input_branches.1.layers.0.scn.bn.weight", "input_branches.1.layers.0.scn.bn.bias", "input_branches.1.layers.0.scn.bn.running_mean", "input_branches.1.layers.0.scn.bn.running_var", "input_branches.1.layers.0.tcn.conv.weight", "input_branches.1.layers.0.tcn.conv.bias", "input_branches.1.layers.0.tcn.bn.weight", "input_branches.1.layers.0.tcn.bn.bias", "input_branches.1.layers.0.tcn.bn.running_mean", "input_branches.1.layers.0.tcn.bn.running_var", "input_branches.1.layers.1.edge", "input_branches.1.layers.1.scn.conv_down.weight", "input_branches.1.layers.1.scn.conv_down.bias", "input_branches.1.layers.1.scn.bn_down.weight", "input_branches.1.layers.1.scn.bn_down.bias", "input_branches.1.layers.1.scn.bn_down.running_mean", "input_branches.1.layers.1.scn.bn_down.running_var", "input_branches.1.layers.1.scn.conv.gcn.weight", "input_branches.1.layers.1.scn.conv.gcn.bias", "input_branches.1.layers.1.scn.bn.weight", "input_branches.1.layers.1.scn.bn.bias", "input_branches.1.layers.1.scn.bn.running_mean", "input_branches.1.layers.1.scn.bn.running_var", "input_branches.1.layers.1.scn.conv_up.weight", "input_branches.1.layers.1.scn.conv_up.bias", "input_branches.1.layers.1.scn.bn_up.weight", "input_branches.1.layers.1.scn.bn_up.bias", "input_branches.1.layers.1.scn.bn_up.running_mean", "input_branches.1.layers.1.scn.bn_up.running_var", "input_branches.1.layers.1.tcn.conv_down.weight", "input_branches.1.layers.1.tcn.conv_down.bias", "input_branches.1.layers.1.tcn.bn_down.weight", "input_branches.1.layers.1.tcn.bn_down.bias", "input_branches.1.layers.1.tcn.bn_down.running_mean", "input_branches.1.layers.1.tcn.bn_down.running_var", "input_branches.1.layers.1.tcn.conv.weight", "input_branches.1.layers.1.tcn.conv.bias", "input_branches.1.layers.1.tcn.bn.weight", "input_branches.1.layers.1.tcn.bn.bias", "input_branches.1.layers.1.tcn.bn.running_mean", "input_branches.1.layers.1.tcn.bn.running_var", "input_branches.1.layers.1.tcn.conv_up.weight", "input_branches.1.layers.1.tcn.conv_up.bias", "input_branches.1.layers.1.tcn.bn_up.weight", "input_branches.1.layers.1.tcn.bn_up.bias", "input_branches.1.layers.1.tcn.bn_up.running_mean", "input_branches.1.layers.1.tcn.bn_up.running_var", "input_branches.1.layers.2.edge", "input_branches.1.layers.2.scn.residual.0.weight", "input_branches.1.layers.2.scn.residual.0.bias", "input_branches.1.layers.2.scn.residual.1.weight", "input_branches.1.layers.2.scn.residual.1.bias", "input_branches.1.layers.2.scn.residual.1.running_mean", "input_branches.1.layers.2.scn.residual.1.running_var", "input_branches.1.layers.2.scn.conv_down.weight", "input_branches.1.layers.2.scn.conv_down.bias", "input_branches.1.layers.2.scn.bn_down.weight", "input_branches.1.layers.2.scn.bn_down.bias", "input_branches.1.layers.2.scn.bn_down.running_mean", "input_branches.1.layers.2.scn.bn_down.running_var", "input_branches.1.layers.2.scn.conv.gcn.weight", "input_branches.1.layers.2.scn.conv.gcn.bias", "input_branches.1.layers.2.scn.bn.weight", "input_branches.1.layers.2.scn.bn.bias", "input_branches.1.layers.2.scn.bn.running_mean", "input_branches.1.layers.2.scn.bn.running_var", "input_branches.1.layers.2.scn.conv_up.weight", "input_branches.1.layers.2.scn.conv_up.bias", "input_branches.1.layers.2.scn.bn_up.weight", "input_branches.1.layers.2.scn.bn_up.bias", "input_branches.1.layers.2.scn.bn_up.running_mean", "input_branches.1.layers.2.scn.bn_up.running_var", "input_branches.1.layers.2.tcn.conv_down.weight", "input_branches.1.layers.2.tcn.conv_down.bias", "input_branches.1.layers.2.tcn.bn_down.weight", "input_branches.1.layers.2.tcn.bn_down.bias", "input_branches.1.layers.2.tcn.bn_down.running_mean", "input_branches.1.layers.2.tcn.bn_down.running_var", "input_branches.1.layers.2.tcn.conv.weight", "input_branches.1.layers.2.tcn.conv.bias", "input_branches.1.layers.2.tcn.bn.weight", "input_branches.1.layers.2.tcn.bn.bias", "input_branches.1.layers.2.tcn.bn.running_mean", "input_branches.1.layers.2.tcn.bn.running_var", "input_branches.1.layers.2.tcn.conv_up.weight", "input_branches.1.layers.2.tcn.conv_up.bias", "input_branches.1.layers.2.tcn.bn_up.weight", "input_branches.1.layers.2.tcn.bn_up.bias", "input_branches.1.layers.2.tcn.bn_up.running_mean", "input_branches.1.layers.2.tcn.bn_up.running_var", "input_branches.2.A", "input_branches.2.bn.weight", "input_branches.2.bn.bias", "input_branches.2.bn.running_mean", "input_branches.2.bn.running_var", "input_branches.2.layers.0.edge", "input_branches.2.layers.0.scn.conv.gcn.weight", "input_branches.2.layers.0.scn.conv.gcn.bias", "input_branches.2.layers.0.scn.bn.weight", "input_branches.2.layers.0.scn.bn.bias", "input_branches.2.layers.0.scn.bn.running_mean", "input_branches.2.layers.0.scn.bn.running_var", "input_branches.2.layers.0.tcn.conv.weight", "input_branches.2.layers.0.tcn.conv.bias", "input_branches.2.layers.0.tcn.bn.weight", "input_branches.2.layers.0.tcn.bn.bias", "input_branches.2.layers.0.tcn.bn.running_mean", "input_branches.2.layers.0.tcn.bn.running_var", "input_branches.2.layers.1.edge", "input_branches.2.layers.1.scn.conv_down.weight", "input_branches.2.layers.1.scn.conv_down.bias", "input_branches.2.layers.1.scn.bn_down.weight", "input_branches.2.layers.1.scn.bn_down.bias", "input_branches.2.layers.1.scn.bn_down.running_mean", "input_branches.2.layers.1.scn.bn_down.running_var", "input_branches.2.layers.1.scn.conv.gcn.weight", "input_branches.2.layers.1.scn.conv.gcn.bias", "input_branches.2.layers.1.scn.bn.weight", "input_branches.2.layers.1.scn.bn.bias", "input_branches.2.layers.1.scn.bn.running_mean", "input_branches.2.layers.1.scn.bn.running_var", "input_branches.2.layers.1.scn.conv_up.weight", "input_branches.2.layers.1.scn.conv_up.bias", "input_branches.2.layers.1.scn.bn_up.weight", "input_branches.2.layers.1.scn.bn_up.bias", "input_branches.2.layers.1.scn.bn_up.running_mean", "input_branches.2.layers.1.scn.bn_up.running_var", "input_branches.2.layers.1.tcn.conv_down.weight", "input_branches.2.layers.1.tcn.conv_down.bias", "input_branches.2.layers.1.tcn.bn_down.weight", "input_branches.2.layers.1.tcn.bn_down.bias", "input_branches.2.layers.1.tcn.bn_down.running_mean", "input_branches.2.layers.1.tcn.bn_down.running_var", "input_branches.2.layers.1.tcn.conv.weight", "input_branches.2.layers.1.tcn.conv.bias", "input_branches.2.layers.1.tcn.bn.weight", "input_branches.2.layers.1.tcn.bn.bias", "input_branches.2.layers.1.tcn.bn.running_mean", "input_branches.2.layers.1.tcn.bn.running_var", "input_branches.2.layers.1.tcn.conv_up.weight", "input_branches.2.layers.1.tcn.conv_up.bias", "input_branches.2.layers.1.tcn.bn_up.weight", "input_branches.2.layers.1.tcn.bn_up.bias", "input_branches.2.layers.1.tcn.bn_up.running_mean", "input_branches.2.layers.1.tcn.bn_up.running_var", "input_branches.2.layers.2.edge", "input_branches.2.layers.2.scn.residual.0.weight", "input_branches.2.layers.2.scn.residual.0.bias", "input_branches.2.layers.2.scn.residual.1.weight", "input_branches.2.layers.2.scn.residual.1.bias", "input_branches.2.layers.2.scn.residual.1.running_mean", "input_branches.2.layers.2.scn.residual.1.running_var", "input_branches.2.layers.2.scn.conv_down.weight", "input_branches.2.layers.2.scn.conv_down.bias", "input_branches.2.layers.2.scn.bn_down.weight", "input_branches.2.layers.2.scn.bn_down.bias", "input_branches.2.layers.2.scn.bn_down.running_mean", "input_branches.2.layers.2.scn.bn_down.running_var", "input_branches.2.layers.2.scn.conv.gcn.weight", "input_branches.2.layers.2.scn.conv.gcn.bias", "input_branches.2.layers.2.scn.bn.weight", "input_branches.2.layers.2.scn.bn.bias", "input_branches.2.layers.2.scn.bn.running_mean", "input_branches.2.layers.2.scn.bn.running_var", "input_branches.2.layers.2.scn.conv_up.weight", "input_branches.2.layers.2.scn.conv_up.bias", "input_branches.2.layers.2.scn.bn_up.weight", "input_branches.2.layers.2.scn.bn_up.bias", "input_branches.2.layers.2.scn.bn_up.running_mean", "input_branches.2.layers.2.scn.bn_up.running_var", "input_branches.2.layers.2.tcn.conv_down.weight", "input_branches.2.layers.2.tcn.conv_down.bias", "input_branches.2.layers.2.tcn.bn_down.weight", "input_branches.2.layers.2.tcn.bn_down.bias", "input_branches.2.layers.2.tcn.bn_down.running_mean", "input_branches.2.layers.2.tcn.bn_down.running_var", "input_branches.2.layers.2.tcn.conv.weight", "input_branches.2.layers.2.tcn.conv.bias", "input_branches.2.layers.2.tcn.bn.weight", "input_branches.2.layers.2.tcn.bn.bias", "input_branches.2.layers.2.tcn.bn.running_mean", "input_branches.2.layers.2.tcn.bn.running_var", "input_branches.2.layers.2.tcn.conv_up.weight", "input_branches.2.layers.2.tcn.conv_up.bias", "input_branches.2.layers.2.tcn.bn_up.weight", "input_branches.2.layers.2.tcn.bn_up.bias", "input_branches.2.layers.2.tcn.bn_up.running_mean", "input_branches.2.layers.2.tcn.bn_up.running_var". size mismatch for input_branches.0.bn.weight: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([5]). size mismatch for input_branches.0.bn.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([5]). size mismatch for input_branches.0.bn.running_mean: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([5]). size mismatch for input_branches.0.bn.running_var: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([5]). size mismatch for input_branches.0.layers.0.scn.conv.gcn.weight: copying a param with shape torch.Size([192, 3, 1, 1]) from checkpoint, the shape in current model is torch.Size([192, 5, 1, 1]). size mismatch for main_stream.0.scn.residual.0.weight: copying a param with shape torch.Size([128, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 96, 1, 1]). size mismatch for main_stream.0.scn.conv_down.weight: copying a param with shape torch.Size([16, 32, 1, 1]) from checkpoint, the shape in current model is torch.Size([16, 96, 1, 1]).

Please, can you advice on where can I find the weights & model which correspond to each other? Thank you!

torch-geometric version

root@Ic479e91c301201ee1:/hy-nas/GaitGraph2-main/GaitGraph# python3 gaitgraph_casia_b.py fit --config configs/casia_b.yaml
Traceback (most recent call last):
File "gaitgraph_casia_b.py", line 8, in
from torch_geometric.data import Data
File "/usr/local/lib/python3.8/dist-packages/torch_geometric/init.py", line 4, in
import torch_geometric.data
File "/usr/local/lib/python3.8/dist-packages/torch_geometric/data/init.py", line 1, in
from .data import Data
File "/usr/local/lib/python3.8/dist-packages/torch_geometric/data/data.py", line 9, in
from torch_sparse import SparseTensor
File "/usr/local/lib/python3.8/dist-packages/torch_sparse/init.py", line 19, in
torch.ops.load_library(spec.origin)
File "/usr/local/lib/python3.8/dist-packages/torch/_ops.py", line 104, in load_library
ctypes.CDLL(path)
File "/usr/lib/python3.8/ctypes/init.py", line 373, in init
self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.8/dist-packages/torch_sparse/_version_cuda.so: undefined symbol: ZN3c106detail23torchInternalAssertFailEPKcS2_jS2_S2

error in running code in caisa-b

I use this command python3 gaitgraph_casia_b.py fit --config configs/casia_b.yaml
but it always give me error reply

File "gaitgraph_casia_b.py", line 77, in validation_step
  y_hat = self(x)
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
  return forward_call(*input, **kwargs)
File "gaitgraph_casia_b.py", line 64, in forward
  return self.backbone(x)[0]
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
  return forward_call(*input, **kwargs)
File "/home/fuyang/project/keypoint/GaitGraph2/GaitGraph2-main/GaitGraph/models/ResGCNv1/nets.py", line 66, in forward
  x_cat.append(branch(x[:, i]))
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
  return forward_call(*input, **kwargs)
File "/home/fuyang/project/keypoint/GaitGraph2/GaitGraph2-main/GaitGraph/models/ResGCNv1/nets.py", line 24, in forward
  x = self.bn(x)
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
  return forward_call(*input, **kwargs)
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/modules/batchnorm.py", line 168, in forward
  return F.batch_norm(
File "/home/fuyang/anaconda3/envs/gaitgraph/lib/python3.8/site-packages/torch/nn/functional.py", line 2421, in batch_norm
  return torch.batch_norm(
RuntimeError: running_mean should contain 5 elements not 15

Running problem of

Hi, I use the command of python3 gaitgraph_casia_b.py fit --config configs/casia_b.yaml but get error
Hope you can help me to solve it
Thank you
Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "gaitgraph_casia_b.py", line 283, in
cli_main()
File "gaitgraph_casia_b.py", line 273, in cli_main
LightningCLI(
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 566, in init
self._run_subcommand(self.subcommand)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/utilities/cli.py", line 837, in _run_subcommand
fn(**fn_kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 768, in fit
self._call_and_handle_interrupt(
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 721, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 809, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1234, in _run
results = self._run_stage()
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1321, in _run_stage
return self._run_train()
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1343, in _run_train
self._run_sanity_check()
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1411, in _run_sanity_check
val_loop.run()
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 154, in advance
dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 127, in advance
output = self._evaluation_step(**kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 222, in _evaluation_step
output = self.trainer._call_strategy_hook("validation_step", *kwargs.values())
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1763, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 344, in validation_step
return self.model.validation_step(*args, **kwargs)
File "gaitgraph_casia_b.py", line 77, in validation_step
y_hat = self(x)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "gaitgraph_casia_b.py", line 64, in forward
return self.backbone(x)[0]
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/fuyang/KeyPoint/GaitGraph2-main/GaitGraph/models/ResGCNv1/nets.py", line 65, in forward
x_cat.append(branch(x[:, i]))
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/fuyang/KeyPoint/GaitGraph2-main/GaitGraph/models/ResGCNv1/nets.py", line 24, in forward
x = self.bn(x)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/modules/batchnorm.py", line 168, in forward
return F.batch_norm(
File "/home/fuyang/anaconda3/envs/HID/lib/python3.8/site-packages/torch/nn/functional.py", line 2421, in batch_norm
return torch.batch_norm(
RuntimeError: running_mean should contain 5 elements not 15

KeyError: 'test_loop'

I am getting the error during testing the pretrained model
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Traceback (most recent call last):
File "D:\FYP\GaitGraph2\GaitGraph\gaitgraph_casia_b.py", line 317, in
cli_main()
File "D:\FYP\GaitGraph2\GaitGraph\gaitgraph_casia_b.py", line 307, in cli_main
LightningCLI(
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\cli.py", line 353, in init
self._run_subcommand(self.subcommand)
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\cli.py", line 642, in _run_subcommand
fn(**fn_kwargs)
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 737, in test
return call._call_and_handle_interrupt(
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\call.py", line 42, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 780, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 966, in _run
self._checkpoint_connector.restore_training_state()
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py", line 292, in restore_training_state
self.restore_loops()
File "C:\Users\Pc\miniconda3\envs\hrnet\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py", line 336, in restore_loops
self.trainer.test_loop.load_state_dict(state_dict["test_loop"])
KeyError: 'test_loop'

RuntimeError: running_mean should contain 5 elements not 15

How to solve the above problems

GaitGraph not found error

when i run the script it shows the following error, how can i resolve it?

Traceback (most recent call last):
File "E:\gait-recognition\GaitGraph2-main\GaitGraph\gaitgraph_casia_b.py", line 13, in
from GaitGraph import cli_logo
ModuleNotFoundError: No module named 'GaitGraph'

meaning of "connect_joint"and "center" in graph.py

Hi, I wonder two parameters in graph.py. What's the meaning of "connect_joint" and "center", how to set them?

Uncertainty on selecting oumvlp-pose dataset - OpenPose or AlphaPose?

I am facing some uncertainty in deciding which dataset to use since both OpenPose-based and AlphaPose-based datasets are available in the given link. I would appreciate any guidance or recommendations from the community on which dataset is better suited for my use case. Thank you in advance for your help.

GPU memory requirement

Hi, I'm wondering how much memory is required?

I've lower the batch_size to 1 and even decrease the embedding_layer

but still OOM with my RTX3080 (10G)