Awesome-LLM-for-Autonomous-Driving

This repository is dedicated to the collection of research papers on LLM for Autonomous Driving. We aim to continuously update it with the latest and most cutting-edge research in the field. This repository is maintained by the Intelligent Vehicle Institute of SJTU.

We invite you to follow and star this repository! If you are interested in further communication or discussion, please don't hesitate to contact me at [email protected] and [email protected].

📚 Survey

A Survey for Foundation Models in Autonomous Driving. 2024.02
World Models for Autonomous Driving: An Initial Survey. 2024.05
A Survey of Large Lanuguage Models for Autonomous Driving. 2023.11
A Survey on Multimodal Large Language Models for Autonomous Driving. [WACV 2024]
Vision Language Models in Autonomous Driving and Intelligent Transportation Systems.2023.10
Towards Knowledge-driven Autonomous Driving. 2023.12

📑 LLM for Autonomous Vehicle Perception

Language-Image Models with 3D Understanding. 2024.05
LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning. 2024.04
OpenAnnotate2: Multi-modal Auto-annotating for Autonomous Driving. [IEEE TIV]
AccidentBlip2: Accident Detection With Multi-View MotionBlip2. 2024.04
AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving. 2024.03
HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving. 2023.09
Language Prompt for Autonomous Driving. 2023.09
GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation. [ICLR 2024]
SPOT: Scalable 3D Pre-training via Occupancy Prediction for Autonomous Driving.

📑 LLM for Autonomous Vehicle Prediction

RAG-based Explainable Prediction of Road Users Behaviors for Automated Driving using Knowledge Graphs and Large Language Models. 2024.05
Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models. 2024.05
LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Models. 2024.03
Large Language Models Powered Context-aware Motion Prediction. 2024.03
GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models. 2023.12
LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving. 2024.03
GPT-4V Takes the Wheel: Promises and Challenges for Pedestrian Behavior Prediction. 2024.01
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving. 2023.09
MTD-GPT: A Multi-Task Decision-Making GPT Model for Autonomous Driving at Unsignalized Intersections. 2023.07

📑 LLM for Autonomous Vehicle Decision

DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences. [VLADR 2024]
Enhancing Q-Learning with Large Language Model Heuristics. 2024.05
Large Language Models as Decision Makers for Autonomous Driving
AGENTSCODRIVER: Large Language Model Empowered Collaborative Driving with Lifelong Learning. 2024.04
Multi-Modal GPT-4 Aided Action Planning and Reasoning for Self-driving Vehicles. [ICASSP 2024]
Drive Like a Human: Rethinking Autonomous Driving with Large Language Models.2023.07
DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models. [ICLR 2024]
Large Language Models for Autonomous Driving:Real-World Experiments.
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving. 2023.10
MTD-GPT: A Multi-Task Decision-Making GPT Model for Autonomous Driving at Unsignalized Intersections. 2023.07

📑 LLM for Autonomous Vehicle Planning

OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception,Reasoning and Planning. 2024.05
NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions. 2024.05
AGENTSCODRIVER: Large Language Model Empowered Collaborative Driving with Lifelong Learning. 2024.04
Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs. 2024.04
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models. 2024.02
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving. 2023.12
Driving Everywhere with Large Language Model Policy Adaptation. 2024.04
Feedback-Guided Autonomous Driving. [CVPR 2024]
LangProp: A code optimization framework using Large Language Models applied to driving. 2024.05
LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning. 2023.12
DriveLM: Driving with Graph Visual Question Answering. [CVPR 2024 Autonomous Driving Challenge]
LMDrive: Closed-Loop End-to-End Driving with Large Language Models. [CVPR 2024]
Empowering Autonomous Driving with Large Language Models: A Safety Perspective. 2023.12
ADriver-I: A General World Model for Autonomous Driving. 2023.11
A Language Agent for Autonomous Driving. 2023.11
ChatGPT as Your Vehicle Co-Pilot: An Initial Attempt. 2023.10
Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles. 2023.10
DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model. 2023.10
GPT-DRIVER: LEARNING TO DRIVE WITH GPT. 2023.10

🔖 License

Awesome-LLM-for-Autonomous-Driving is released under the Apache 2.0 license.

nyx990217 / awesome-llm-for-autonomous-driving Goto Github PK