autovision-cloud / ssl-lanes

[CoRL-2022] SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving

SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving

By Prarthana Bhattacharyya, Chengjie Huang and Krzysztof Czarnecki.

We provide code support for the paper: SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving.
Our code is based on Lane-GCN, which is a clean open-sourced project for motion forecasting methods.

Overview

In this study, we report the first systematic exploration and assessment of incorporating self-supervision into motion forecasting. We first propose to investigate four novel self-supervised learning tasks for motion forecasting with theoretical rationale and quantitative and qualitative comparisons on the challenging large-scale Argoverse dataset. Secondly, we point out that our auxiliary SSL-based learning setup not only outperforms forecasting methods which use transformers, complicated fusion mechanisms and sophisticated online dense goal candidate optimization algorithms in terms of performance accuracy, but also has low inference time and architectural complexity. Lastly, we conduct several experiments to understand why SSL improves motion forecasting.

Results

Quantitative Results

For this repository, the expected performance on Argoverse 1 validation set is:

Models	minADE	minFDE	MR
Baseline	0.73	1.12	11.07
Lane-Masking	0.70	1.02	8.82
Distance to Intersection	0.71	1.04	8.93
Maneuver Classification	0.72	1.05	9.36
Success/Failure Classification	0.70	1.01	8.59

Qualitative Results

Pretrained Models

We provide the pretrained checkpoints for the proposed above-mentioned models in checkpoints/.

Inference validation set for metrics

To evaluate the prediction performance, run:

python test.py -m lanegcn --weight=lane_masking.ckpt --split=val

Usage

1. Clone this repository:

git clone https://github.com/AutoVision-cloud/SSL-Lanes
cd SSL-Lanes

2. Following is an example of create environment from scratch with anaconda, you can use pip as well:

conda create --name lanegcn python=3.7
conda activate lanegcn
conda install pytorch==1.5.1 torchvision cudatoolkit=10.2 -c pytorch # pytorch=1.5.1 when the code is release

# install argoverse api
pip install  git+https://github.com/argoai/argoverse-api.git

# install others dependancy
pip install scikit-image IPython tqdm ipdb

3. Install Horovod and mpi4py for distributed training. Horovod is more efficient than nn.DataParallel for mulit-gpu training and easier to use than nn.DistributedDataParallel. Before install horovod, make sure you have openmpi installed (sudo apt-get install -y openmpi-bin).

pip install mpi4py

# install horovod with GPU support, this may take a while
HOROVOD_GPU_OPERATIONS=NCCL pip install horovod==0.19.4

# if you have only SINGLE GPU, install for code-compatibility
pip install horovod

if you have any issues regarding horovod, please refer to horovod github

4. Download Argoverse Motion Forecasting Dataset v1.1. After downloading and extracting the tar.gz files, the dataset directory should be organized as follows:

/path/to/dataset_root/
├── train/
|   └── data/
|       ├── 1.csv
|       ├── 2.csv
|       ├── ...
└── val/
    └── data/
        ├── 1.csv
        ├── 2.csv
        ├── ...

5. Download the processed data.

bash get_data.sh

Training

Training with Horovod-multigpus

Copy the contents of the pretext task into ssl_pretext_tasks/LaneGCN as follows:

cp -r ssl_pretext_tasks/lane_masking/final_task_masking/ ssl_pretext_tasks/LaneGCN/

# single node with 4 gpus
horovodrun -np 4 -H localhost:4 python /path/to/train.py -m lanegcn

Citation

If you find this project useful in your research, please consider starring the repository and citing:

@misc{bhattacharyya2022ssllanes,
      title={SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Driving}, 
      author={Prarthana Bhattacharyya, Chengjie Huang and Krzysztof Czarnecki},
      year={2022},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

• LaneGCN
• SelfTask-GNN
• SS-GCN