tteepe / earlybird

Official Code for "EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View"

Early Bird 🦅

EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View

Torben Teepe, Philipp Wolters, Johannes Gilg, Fabian Herzog, Gerhard Rigoll

Usage

Getting Started

1. Install PyTorch with CUDA support

   pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118

2. Install mmcv with CUDA support

   pip install mmcv==2.0.0 -f https://download.openmmlab.com/mmcv/dist/cu118/torch2.1/index.html

3. Install remaining dependencies

   pip install -r requirements.txt

Training

python main.py fit -c configs/t_fit.yml \
    -c configs/d_{multiviewx,wildtrack}.yml

Testing

python main.py test -c model_weights/config.yaml \
    --ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt

Acknowledgement

• Simple-BEV: Adam W. Harley
• MVDeTr: Yunzhong Hou

Cite

If you use EarlyBird, please use the following BibTeX entry.

@InProceedings{teepe2023earlybird,
    author    = {Teepe, Torben and Wolters, Philipp and Gilg, Johannes and Herzog, Fabian and Rigoll, Gerhard},
    title     = {Early{B}ird: Early-Fusion for Multi-View Tracking in the Bird's Eye View},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops},
    month     = {January},
    year      = {2024},
    pages     = {102-111}
}

earlybird's Issues

Results

Dear authors,

I am trying to test the model and see the results, and I tried with different backbone networks [resnet18, resnet50, swin-t] but the results I get are different from the ones mentioned in the paper, for example I am getting a low track/MOTP score. I would appreciate your advice regarding this matter and thank you for your time!

Best wishes,

Where did you get "self.worldcoord_from_worldgrid_mat" from?

Thank you for your great work, it helps us a lot for our project. We really appreciate it.

We do not know how fixed parameters or matrices was designed? From the config of each dataset or manually setting?
For example, "self.worldcoord_from_worldgrid_mat = np.array([[0, 2.5, -300], [2.5, 0, -900], [0, 0, 1]])" in line 23 of "EarlyBird/EarlyBird/datasets/wildtrack_dataset.py". How to set self.worldcoord_from_worldgrid_mat like that.

And:
grid_y = pos % 480
grid_x = pos // 480
How to know the position divided by 480 but not other number.

Model Weights for Testing

Thank you for the great paper.

I have a question about whether the model weights are going to be released.

Best regards,

Weights

Hello! Thank you for your excellent paper!

In my reproduction, the code "resnet = torchvision.models.resnet18(weights=torchvision.models.ResNet18_Weights.DEFAULT)" is reporting error "AttributeError: module ' AttributeError: module ' torchvision.models' has no attribute 'ResNet18_Weights'", it was solved by changing it to "resnet = torchvision.models.resnet18(pretrained=True)".

After training, run "python main.py test -c model_weights/config.yaml --ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt" command, I got error "error: Parser key "config": File does not exist: '/ home/zhangkeyu/EarlyBird/EarlyBird/model_weights/config.yaml'".
Is there any file missing? Or what should I do?

I hope to reply as soon as possible, this is very important to me, thanks!

Translated with DeepL.com (free version)