hyunjs / sgt

[WACV-2023] Detection Recovery in Online Multi-Object Tracking with Sparse Graph Tracker

Sparse Graph Tracker (SGT)

Official code for Sparse Graph Tracker (SGT) based on the Detectron2 framework. Please feel free to leave an ISSUE or send me an email ([email protected]).

News

• (2022.10.11) Our paper is accepted WACV 2023! (arxiv paper will be updated soon)
• (2022.10.06) Code and pretrained weights are released!

Installation

• Please refer INSTALL.md for the details

Dataset Setup

• Please refer DATASET.md for the details

Model Zoo

• download model weights

• Please modify the path of checkpoints in the config file based on your checkpoint directory

MOT17

Name	Dataset	HOTA	MOTA	IDF1	Download
SGT	MOT17	58.2	73.2	70.2	model
SGT	MOT17 + CrowdHuman	60.8	76.4	72.8	model

MOT20

Name	Dataset	HOTA	MOTA	IDF1	Download
SGT	MOT20	51.6	64.5	62.7	model
SGT	MOT20 + CrowdHuman	57.0	72.8	70.6	model

HiEve

Name	Dataset	MOTA	IDF1	Download
SGT	HiEve	47.2	53.7	model

How to run?

Train

python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 2 OUTPUT_DIR /root/sgt_output/mot17_val/dla34_mot17-CH

Inference

python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 1 --eval-only OUTPUT_DIR /root/sgt_output/mot17_test/dla34_mot17-CH

Visualization

## GT
python projects/Datasets/MOT/vis/vis_gt.py --data-root <$DATA_ROOT> --register-data-name <e.g., mot17_train> 
python projects/Datasets/MOT/vis/vis_gt.py --data-root <$DATA_ROOT> --register-data-name <e.g., mix_crowdhuman_train> --no-video-flag 


## model output
python projects/Datasets/MOT/vis/vis_seq_from_txt_result.py --data-root <$DATA_ROOT> --result-dir <$OUTPUT_DIR> --data-name {mot17, mot20, hieve, mot17_sub, mot20_sub} --tgt-split {val,test}

Motivation

Pipeline

MOT Benchmark Results

Ablation Experiment Results

Visualization

License

Code of SGT is licensed under the CC-BY-NC 4.0 license and free for research and academic purpose. SGT is based on the framework Detectron2 which is released under the Apache 2.0 license and the detector CenterNet which is released under the MIT license. This codebase also provides Detectron2 version of FairMOT which is released under the MIT license.

Citation

@inproceedings{hyun2023detection,
  title={Detection recovery in online multi-object tracking with sparse graph tracker},
  author={Hyun, Jeongseok and Kang, Myunggu and Wee, Dongyoon and Yeung, Dit-Yan},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={4850--4859},
  year={2023}
}

sgt's Issues

Code release ETA

Hi, found your work really interesting. Is there an ETA for the code release? Thanks!

How to test?

Thank you for sharing your code. However, I met some error when testing.
assert len(dicts), "Dataset '{}' is empty!".format(dataset_name)
AssertionError: Dataset 'mot17_sub_val' is empty!

I will appreciate that you can tell me how to prepared test/val datasets and test with your code.

Even I try to test with the training data it still has some errors.AttributeError: Attribute 'detector' does not exist in the metadata of dataset 'mix_mot17_sub_train'. Available keys are dict_keys(['name', 'thing_classes', 'thing_colors', 'split', 'evaluator_type', 'root_dir', 'has_ids_flag', 'is_consecutive_flag', 'dense_video_flag']).

Hope to get reply from you.

About the inference

I fit my own dataset into SGT but in the inference step the detection across 3 datasets of 442 images are only one left. Why is this happening and how can I fix this? and the eval results saed to csv format is at which place, I cannot find it.. The serializing is for 442 images and the size of the serialized is only 0.09MB

code release

Hi, found your work really interesting. Is there an ETA for the code release? Thanks!

[04/05 10:36:05 projects.Datasets.MIX.builtin]: Build dataset dict from mot17_sub.train [04/05 10:36:06 projects.Datasets.MIX.builtin]: % bboxes below visibility threshold 0.0 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-02-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-04-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-05-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-09-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-10-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-11-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-13-sdp : 0.00 [04/05 10:36:06 projects.Datasets.MOT.build]: Removed 0 images with no usable annotations. 2657 images left.

When I run python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /data/liuqigui/project/dataset --num-gpus 1 OUTPUT_DIR /data/liuqigui/project/SGT-main/sgt_output/mot17_train/dla34_mot17-CH2， this problem happened.

[04/05 10:36:05 projects.Datasets.MIX.builtin]: Build dataset dict from mot17_sub.train
[04/05 10:36:06 projects.Datasets.MIX.builtin]: % bboxes below visibility threshold 0.0
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-02-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-04-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-05-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-09-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-10-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-11-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MIX.builtin]: mot17-13-sdp : 0.00
[04/05 10:36:06 projects.Datasets.MOT.build]: Removed 0 images with no usable annotations. 2657 images left.

I don't know why this problem occurred. The dataset was also downloaded from yours. The CrowdHuman dataset also has this problem.

no ctdet_coco_dla_2x_converted.pth file and the eval phase

hello,
i didin't find the file WEIGHTS: "/root/checkpoints/ctdet_coco_dla_2x_converted.pth" for /SGT/projects/SGT/configs/hieve/sgt_dla34.yaml
in download model weights

_evaluate_predictions_on_coco( TypeError: _evaluate_predictions_on_coco() got an unexpected keyword argument 'use_fast_impl'

After 5 epoch of training, an error occurs when the model is saved to test the effect.

python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 1 OUTPUT_DIR /root/sgt_output/mot17_val/dla34_mot17-CH

It doesn't work well

According to your file configuration, you can't run the effect as you said, the result is as follows, two GPUs have been used, and the following result is the effect of one GPU
python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 1 OUTPUT_DIR /root/sgt_output/mot17_val/dla34_mot17-CH

no detectron2 file

i have a question! When python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 2 OUTPUT_DIR /root/sgt_output/mot17_val/dla34_mot17-CH, the computer shows Traceback (most recent call last):
File "FairMOT-base/src/train_net.py", line 9, in
import detectron2.utils.comm as comm
ModuleNotFoundError: No module named 'detectron2'

where is the detectron2? could you send me the detectron2 file?

KeyError: 'Non-existent config key: SGT'

Hello, when I execute python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 1 -- eval-only OUTPUT_DIR /root/sgt_output/mot17_test/dla34_mot17-CH command, I encountered KeyError: 'Non-existent config key: SGT'. What should I do to fix it?
thank you!

checkpoint warning

I meet this warning when i training the MOT17. what's this?

I have a problem with such an error when I execute the python projects/SGT/train_net.py --config-file projects/SGT/configs/MOT17/sgt_dla34.yaml --data-dir /root/datasets --num-gpus 2 OUTPUT_DIR /root/sgt_output/mot17_val/dla34_mot17-CHcommand

Traceback (most recent call last):
File "projects/SGT/train_net.py", line 70, in
launch(
File "/home1/liuqigui/project/SGT-main/detectron2/detectron2/engine/launch.py", line 69, in launch
mp.start_processes(
File "/home/wguo/anaconda3/envs/sgt1/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
while not context.join():
File "/home/wguo/anaconda3/envs/sgt1/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 150, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/home/wguo/anaconda3/envs/sgt1/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
fn(i, *args)
File "/home1/liuqigui/project/SGT-main/detectron2/detectron2/engine/launch.py", line 123, in _distributed_worker
main_func(*args)
File "/home1/liuqigui/project/SGT-main/projects/SGT/train_net.py", line 63, in main
trainer = Trainer(cfg)
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/SGT/sgt/trainer.py", line 40, in init
super().init(cfg)
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/EpochTrainer/epoch_trainer/default_epoch_trainer.py", line 52, in init
data_loader = self.build_train_loader(cfg)
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/SGT/sgt/trainer.py", line 68, in build_train_loader
return build_mix_train_loader(cfg, mapper=mapper)
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/Datasets/MIX/build.py", line 115, in build_mix_train_loader
dataset_dicts, required_meta_data = get_mix_dataset_dicts(cfg.DATASETS.TRAIN,
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/Datasets/MIX/build.py", line 88, in get_mix_dataset_dicts
dataset_dicts, obj_id_set, bbox_stat = DatasetCatalog.get(dataset_name)
File "/home1/liuqigui/project/SGT-main/detectron2/detectron2/data/catalog.py", line 58, in get
return f()
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/Datasets/MIX/builtin.py", line 160, in
DatasetCatalog.register(reg_dataset_name, lambda: load_mix_dataset_dicts(dataset_name, split, root_dir, fill_id_flag,
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/Datasets/MIX/builtin.py", line 275, in load_mix_dataset_dicts
ratio_bbox_below_vis_per_seq = {
File "/home1/liuqigui/project/SGT-main/projects/SGT/../../projects/Datasets/MIX/builtin.py", line 276, in
k: 100 * num_bbox_below_vis_per_seq[k] / (num_bbox_below_vis_per_seq[k] + sum_bbox_per_seq[k]) for k in
ZeroDivisionError: division by zero

What GPU are you using for training models？

code release

Thank you for your great job.
I'm interested in your great work.
I want to know when code will be released.

ZeroDivisionError: division by zero

When I run train_net.py， this problem happened.
This problem occurs when the dataloader is being built.
In builtin.py, when computing the ratio_bbox_below_vis_per_seq, there are many zeros in the denominator.

ratio_bbox_below_vis_per_seq = {
k: 100 * num_bbox_below_vis_per_seq[k] / (num_bbox_below_vis_per_seq[k] + sum_bbox_per_seq[k]) for k in
num_bbox_below_vis_per_seq.keys()}

I don't know what this variable means, or why this problem arises.

error

hello ,
Thank you for your great job,
i have this error when i run the command :
docker run --rm -it -d --shm-size=16G --gpus all -v E:/testSGT/:/var/lib/docke --name sgt -p 3000:22 -p 3001:8081 jshyunaa/sgt:cu113
error :
4047f072d415073b2d4e2b5951c39efa0e4a9152564ce005ca9b1659fe9f80a3
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as ‘legacy’
nvidia-container-cli: initialization error: WSL environment detected but no adapters were found: unknown.

I’m on a Windows 10 Professionnel Version 21H2 home computer, and I have a NVIDIA GeForce GT 710 gpu . This is the output when running docker --version : `Docker version 20.10.17, build 100c701, on both systems (WSL and windows).

code release

Hello ,Thank you for your great job. I'm interested in your work.
I want to know when code will be released.