syscv / tet Goto Github PK

Hi,
Firstly thanks a lot for great work. Your model is basis for my current internship project!

I have done testing on validation split set containing 200 video sequences and I got the same results as mentioned in your paper as shown below:

I want to try testing on small validation split to save time. Hence I only chose first ten validation sequences, make the corresponding annotations using python -m bdd100k.label.to_coco command and run the testing. For the small validation split, the metrics like TETA, LoacA etc were heavily decreased. Even the MOTA is coming up in negative values (see figure below).

I am not sure about the reason for such low metric values for a smaller validation split. I only changed the ann_file, scalabel_gt and img_prefix paths to respective small validation split folder in cem_bdd.py configuration file.

Can you guide me in this issue. Is this the expected output or I am missing something because of which my metrics are coming very low on small validation split. Any suggestion would be helpful for me.

Best Regards,
Rehman

Hi, When I try to test this tracker with pre-trained model. It shows the following errors:

Traceback (most recent call last):
File "tools/test.py", line 164, in
main()
File "tools/test.py", line 111, in main
dataset = build_dataset(cfg.data.test)
File "/home/xd1/.local/lib/python3.8/site-packages/mmdet/datasets/builder.py", line 82, in build_dataset
dataset = build_from_cfg(cfg, DATASETS, default_args)
File "/home/xd1/.local/lib/python3.8/site-packages/mmcv/utils/registry.py", line 55, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
FileNotFoundError: TaoDataset: [Errno 2] No such file or directory: 'data/lvis/annotations/lvis_classes.txt'

I used the following commands:

CONFIG=configs/tao/tracker_swinL_tao.py
CHECKPOINT=pretrain/teter_tao_swinL_1X_20220807_220424.pth
GPUS=1
PORT=25000

export PYTHONPATH="${PYTHONPATH}:$(dirname $0)/.."

python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
    tools/test.py $CONFIG $CHECKPOINT --launcher pytorch ${@:5} \
    --eval track --eval-options resfile_path=results/teter_tao_results/

Could you share the file of lvis_classes.txt with me? Or how to generate this file?
Thank you very much

Hi,
I hope everyone will be fine.

Is it possible to first use detector only and calculate the evaluation metrics e.g. mAP ?
I actually want to conduct an experiment showing that tracker also improves detection results. So I need to run the detection part first and then detection + tracking part.

and the detection results we get from test.py script (see figure below), are they before applying tracker or after?

Kindly guide or share any possible suggestions. It will be much helpful for me.

Hi, I wonder can you provide the checkpoint of the model trained on LVISv0.5 + COCO dataset? many thanks.

Hi, I would like to reproduce the result of QDTrack+CAL in table 4 of the paper, which achieves 30.00(TETA)，50.53(LocA)， 27.36（AssocA） 12.11（ClsA）。Thus I use the rep(https://github.com/SysCV/qdtrack) with the command "sh ./tools/dist_train.sh ./configs/tao/qdtrack_frcnn_r101_fpn_24e_lvis_1230_cls.py 8" and then get the trained model (24th checkpoint) to the next step, with the command "sh ./tools/dist_train.sh ./configs/tao/qdtrack_frcnn_r101_fpn_12e_tao_ft.py 8". In particular, I modified the following code：
1. “num_classes=482“ => “num_classes=1230“
2. the rcnn test config as following:

after completing the two steps training, I got the model with only 25.97(TETA). I wonder can you tell me if I did something wrong? or can tell me how you get the baseline model?

We have reproduced mAP on TAO dataset, and found that the performance is quite low.

The numbers are generated with the block here
, while track3D mAP is calculated with the tao toolkit.

Could you provide the mAP numbers you get on TAO? We need to further verify it there is any bugs here