little-podi / collaborative_perception

This repository is a paper digest of recent advances in collaborative / cooperative / multi-agent perception for V2I / V2V / V2X autonomous driving scenario.

👋Hi there, this is Shenyuan Gao, a 2nd-year Ph.D. student at HKUST

Local Minima

Please find my up-to-date [Resume] to check my current status.

Latent Representation

I was born in the last year of the 20th century (May 17th, 2000). I hope to live until the 22nd century so that my life would span three centuries. The Chinese meanings of each word in my name (高深远) are "High", "Deep", "Far", respectively.

Multi-Stage Pre-Training

• Ph.D. in Electronic and Computer Engineering, 2022-2026 (expected)
  • The Hong Kong University of Science and Technology, under the tutelage of Prof. Jun Zhang (IEEE Fellow)
• B.Eng. in Electronic Information Engineering, 2018-2022
  • Huazhong University of Science and Technology, Advanced Class (GPA 3.9/4.0, ranked 1st)

Selected Checkpoints

• Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability
  • Preprint
• Generalized Predictive Model for Autonomous Driving
  • Accepted by CVPR 2024 as Highlight (Top 2.8%)
• Generalized Relation Modeling for Transformer Tracking
  • Accepted by CVPR 2023
• AiATrack: Attention in Attention for Transformer Visual Tracking
  • Accepted by ECCV 2022

Rewards

• RedBird PhD Scholarship
• Full Postgraduate Scholarship
• Outstanding Graduate
• Outstanding Graduation Thesis
• Outstanding Undergraduate in Terms of Academic Performance (Top 1%)
• National Scholarship (Top 2%)

Region of Interests

• Song: Temple
• Game: Total War: THREE KINGDOMS
• Comic: One Piece

Queries

• School Email: [email protected]
• Personal Email: [email protected]
• Enterprise Email: [email protected]
• WeChat: gsycm0517
• QQ: 917547962

collaborative_perception's Issues

Hi bro, have you try the CoPerception-UAV datasets in the following mothods?

I would like some help with how to convert the datasets into the support type of Opencood.

Question about communication of Where2Comm

Thank you very much for your efforts. I'd want to ask a question about communication efficiency of Where2Comm.
In my opinion, does not need to transmit feature maps. Isn't it sufficient to convey its own pose and bounding box directly? When others receive this, they should convert it to their own coordinate system and then integrate it to fill in the gaps. Transmission of plain text can significantly cut communication costs. So why don't convey pose and bounding box directly？

同学您好，关于when2com和who2com实验复现指标

同学我看您复现的两个方法指标很多是一致的，感觉比较奇怪，我个人在跑https://github.com/GT-RIPL/MultiAgentPerception/中的两个方法的时候也发现了这个问题，甚至有的时候when2com不如who2com

Some materials may be useful

Hi Shenyuan, great work for summarizing collaborative perception! This repo will be very useful!

I have some other materials that may be useful to the community, please consider adding them to your repo if you find them valuable.

[Talk] Cooperative Driving Automation -- Simulation and Perception. Link: https://course.zhidx.com/c/MmQ1YWUyMzM1M2I3YzVlZjE1NzM=
[Library] [ITSC2021[OpenCDA: A simulation framework that supports full-stack cooperative driving automation, including cooperative perception, localization, planning, and control. Link: https://github.com/ucla-mobility/OpenCDA

Question about compression

Hello and thank you for your excellent job. I'm unsure about the compression ratio in Where2Comm and V2X-VIT.
Is it essential to use a compression ratio if Where2comm reduces communication costs by selecting spatially sparse? Will the channel experience a loss as a result of this compression? (The threshold 0.01 already filter out a large portion of the feature mapIs)

Why is the compression rate in V2X-VIT 0x? I'd like to try to reduce communication costs even further, is it permissible to change the compression ratio straight to 32 during training?