ywu40 / flowattack

Attacking Optical Flow (ICCV 2019)

Flow Attack

This is an official repository of

Anurag Ranjan, Joel Janai, Andreas Geiger, Michael J. Black. Attacking Optical Flow. ICCV 2019.

[Project Page] [Arxiv]

Prerequisites

Python3 and pytorch are required. Third party libraries can be installed (in a python3 virtualenv) using:

pip3 install -r requirements.txt

Install custom cuda layers for FlowNet2 using

bash install_flownet2_deps.sh

Preparing training data

Download the KITTI dataset using this script provided on the official website, and then run the following command.

python3 data/prepare_train_data.py /path/to/raw/kitti/dataset/ --dataset-format 'kitti' --dump-root /path/to/resulting/formatted/data/ --width 1280 --height 384 --num-threads 1 --with-gt

For testing optical flow ground truths on KITTI, download KITTI2015 dataset.

Pretrained Models

Download the pretrained models for FlowNetC, FlowNet2, PWC-Net. The pretrained models for SPyNet and Back2Future are provided with this repository.

Generating Adversarial Patches

White-Box Attacks

In the White-Box attacks we optimize a patch for a single network. We use gradient descent as described in the paper. Use the following command to generate an adversarial patch for a specific network architecture using the prepared dataset:

python3 main.py --data [Path to prepared dataset] --kitti-data [Path to KITTI 2015 test set] --flownet [FlowNetS|FlowNetC|FlowNet2|PWCNet|Back2Future|SpyNet] --patch-size 0.10 --name [Name of the experiment]

The patch size is specified in percentage of the training image size (default: 256). All other arguments such as the learning rate, epoch size, etc are set to the values used in our experiments. For details please check main.py

Acknowledgements

• We thank several github users for their contributions which are used in this repository.
  • The code for generating randomized patches and augmentation comes from jhayes14/adversarial-patch.
  • Data preprocessing and KITTI dataloaders code is taken from ClementPinard/SfmLearner-Pytorch/. Optical flow evalution code is taken from anuragranj/cc.
  • FlowNet and FlowNet2 models have been taken from NVIDIA/flownet2-pytorch. PWCNet is taken from NVlabs/PWC-Net.
  • SPyNet implementation is taken from sniklaus/pytorch-spynet.
  • Back2Future implementation is taken from anuragranj/back2future.pytorch.
  • Correlation module is taken from ClementPinard/Pytorch-Correlation-extension.