Papers for Video Anomaly Detection, released codes collections.
Any addition or bug please open an issue, pull requests or e-mail me by [email protected]
Datasets
- UMN
Download link
- UCSD
Download link
- Subway Entrance/Exit
Download link
- CUHK Avenue
Download link
- HD-Avenue Skeleton-based
- ShanghaiTech
Download link
- HD-ShanghaiTech Skeleton-based
- UCF-Crime (Weakly Supervised)
- UCFCrime2Local (subset of UCF-Crime but with spatial annotations.)
Download_link
, Ano-Locality - Spatial Temporal Annotations
Download_link
Background-Bias
- UCFCrime2Local (subset of UCF-Crime but with spatial annotations.)
- Traffic-Train
- Belleview
- Street Scene (WACV 2020) Street Scenes,
Download link
- IITB-Corridor (WACV 2020) Rodrigurs.etl
- XD-Violence (ECCV 2020) XD-Violence
Download link
- ADOC (ACCV 2020) ADOC
Download_link
The Datasets belowed are about Traffic Accidents Anticipating in Dashcam videos or Surveillance videos
-
DAD paper,
Download link
-
A3D paper,
Download link
-
DADA
Download link
-
DoTA
Download_link
-
Iowa DOT
Download_link
-
Driver_Anomaly Project_link
Unsupervised
2016
- [Conv-AE] Learning Temporal Regularity in Video Sequences,
CVPR 16
. Code
2017
- [Hinami.etl] Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge,
ICCV 2017
. (Explainable VAD) - [Stacked-RNN] A revisit of sparse coding based anomaly detection in stacked rnn framework,
ICCV 2017
. code - [ConvLSTM-AE] Remembering history with convolutional LSTM for anomaly detection,
ICME 2017
.Code - [Conv3D-AE] Spatio-Temporal AutoEncoder for Video Anomaly Detection,
ACM MM 17
. - [Unmasking] Unmasking the abnormal events in video,
ICCV 17
. - [DeepAppearance] Deep appearance features for abnormal behavior detection in video
2018
- [FramePred] Future Frame Prediction for Anomaly Detection -- A New Baseline,
CVPR 2018
. code - [ALOOC] Adversarially Learned One-Class Classifier for Novelty Detection,
CVPR 2018
. code - Detecting Abnormality Without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection,
ACM MM 18
.
2019
- [Mem-AE] Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection,
ICCV 2019
.code - [Skeleton-based] Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos,
CVPR 2019
.code - [Object-Centric] Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection,
CVPR 2019
. - [Appearance-Motion Correspondence] Anomaly Detection in Video Sequence with Appearance-Motion Correspondence,
ICCV 2019
.code - [AnoPCN]AnoPCN: Video Anomaly Detection via Deep Predictive Coding Network, ACM MM 2019.
2020
- [Street-Scene] Street Scene: A new dataset and evaluation protocol for video anomaly detection,
WACV 2020
. - [Rodrigurs.etl]) Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection,
WACV 2020
. - [GEPC] Graph Embedded Pose Clustering for Anomaly Detection,
CVPR 2020
.code - [Self-trained] Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection,
CVPR 2020
. - [MNAD] Learning Memory-guided Normality for Anomaly Detection,
CVPR 2020
. code - [Continual-AD]] Continual Learning for Anomaly Detection in Surveillance Videos,
CVPR 2020 Worksop.
- [OGNet] Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm,
CVPR 2020
. code - [Any-Shot] Any-Shot Sequential Anomaly Detection in Surveillance Videos,
CVPR 2020 workshop
. - [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection
ECCV 2020 Spotlight
code - [CDAE]Clustering-driven Deep Autoencoder for Video Anomaly Detection
ECCV 2020
- [VEC]Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events
ACM MM 2020 Oral
code - [ADOC][A Day on Campus - An Anomaly Detection Dataset for Events in a Single Camera]
ACCV 2020
- [CAC]Cluster Attention Contrast for Video Anomaly Detection
ACM MM 2020
- [STC-Graph]Scene-Aware Context Reasoning for Unsupervised Abnormal Event Detection in Videos
ACM MM 2020
2021
- [AMCM]Appearance-Motion Memory Consistency Network for Video Anomaly Detection
AAAI 2021
- [SSMT,Self-Supervised-Multi-Task]Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
CVPR 2021
Weakly-Supervised
2018
- [Sultani.etl] Real-world Anomaly Detection in Surveillance Videos,
CVPR 2018
code
2019
- [GCN-Anomaly] Graph Convolutional Label Noise Cleaner:Train a Plug-and-play Action Classifier for Anomaly Detection,
CVPR 2019
, code - [MLEP] Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies,
IJCAI 2019
code. - [IBL] Temporal Convolutional Network with Complementary Inner Bag Loss For Weakly Supervised Anomaly Detection.
ICIP 19
. - [Motion-Aware] Motion-Aware Feature for Improved Video Anomaly Detection.
BMVC 19
.
2020
- [Siamese] Learning a distance function with a Siamese network to localize anomalies in videos,
WACV 2020
. - [AR-Net] Weakly Supervised Video Anomaly Detection via Center-Guided Discrimative Learning,
ICME 2020
.code - ['XD-Violence'] Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision
ECCV 2020
- [CLAWS] CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection
ECCV 2020
2021
- [MIST] MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection
CVPR 2021
Project Page - [MTN-KMIL] Weakly-supervised Video Anomaly Detection with Contrastive Learning of
Long and Short-range Temporal Features
Arxiv 2021
Code
Supervised
2019
- [Background-Bias]Exploring Background-bias for Anomaly Detection in Surveillance Videos,
ACM MM 19
. - [Ano-Locality]Anomaly locality in video suveillance.
Others
2020
- [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection
ECCV 2020
code
Reviews / Surveys
- An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos, J. Image, 2018.page
- DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY, paper
- Video Anomaly Detection for Smart Surveillance paper
Books
- Outlier Analysis. Charu C. Aggarwal
Specific Scene
Generally, anomaly detection in recent researches are based on the datasets from pedestrian (likes UCSD, Avenue, ShanghaiTech, etc.), or UCF-Crime (real-world anomaly). However some focus on specific scene as follows.
Traffic
CVPR workshop, AI City Challenge series.
First-Person Traffic
Unsupervised Traffic Accident Detection in First-Person Videos, IROS 2019.
Driving
When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos. github
Old-man Fall Down
Fighting/Violence
- Localization Guided Fight Action Detection in Surveillance Videos. ICME 2019.
Social/ Group Anomaly
- Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks, Neurips 2019.
Related Topics:
- Video Representation (Unsupervised Video Representation, reconstruction, prediction etc.)
- Object Detection
- Pedestrian Detection
- Skeleton Detection
- Graph Neural Networks
- GAN
- Action Recognition / Temporal Action Localization
- Metric Learning
- Label Noise Learning
- Cross-Modal/ Multi-Modal
- Dictionary Learning
- One-Class Classification / Novelty Detection / Out-of-Disturibution Detection
- Action Recognition.
- Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events. ACM MM 2020 workshop.
Performance Evaluation Methods
- AUC
- PR-AUC
- Score Gap
- False Alarm Rate on Normal with 0.5 as threshold (Weakly supervised, proposed in CVPR 18)
Performance Comparison on UCF-Crime
Model | Reported on Convference/Journal | Supervised | Feature | Encoder-based | 32 Segments | AUC (%) | [email protected] on Normal (%) |
---|---|---|---|---|---|---|---|
Sultani.etl | CVPR 18 | Weakly | C3D RGB | X | √ | 75.41 | 1.9 |
IBL | ICIP 19 | Weakly | C3D RGB | X | √ | 78.66 | - |
Motion-Aware | BMVC 19 | Weakly | PWC Flow | X | √ | 79.0 | - |
GCN-Anomaly | CVPR 19 | Weakly | TSN RGB | √ | X | 82.12 | 0.1 |
ST-Graph | ACM MM 20 | Un | - | √ | X | 72.7 | |
Background-Bias | ACM MM 19 | Fully | NLN RGB | √ | X | 82.0 | - |
CLAWS | ECCV 20 | Weakly | C3D RGB | √ | X | 83.03 | - |
MIST | CVPR 21 | Weakly | I3D RGB | √ | X | 82.30 | 0.13 |
MTN-KMIL | arxiv 21 | Weakly | I3D RGB | X | √ | 84.03 | - |
Performance Comparison on ShanghaiTech
Model | Reported on Conference/Journal | Supervision | Feature | Encoder-based | AUC(%) | [email protected] (%) |
---|---|---|---|---|---|---|
Conv-AE | CVPR 16 | Un | - | √ | 60.85 | - |
stacked-RNN | ICCV 17 | Un | - | √ | 68.0 | - |
FramePred | CVPR 18 | Un | - | √ | 72.8 | - |
FramePred* | IJCAI 19 | Un | - | √ | 73.4 | - |
Mem-AE | ICCV 19 | Un | - | √ | 71.2 | - |
MNAD | CVPR 20 | Un | - | √ | 70.5 | - |
VEC | ACM MM 20 | Un | - | √ | 74.8 | - |
ST-Graph | ACM MM 20 | Un | - | √ | 74.7 | - |
CAC | ACM MM 20 | Un | - | √ | 79.3 | |
AMMC | AAAI 21 | Un | - | √ | 73.7 | - |
SSMT | CVPR 21 | Un | - | √ | 90.2 | - |
MLEP | IJCAI 19 | 10% test vids with Video Anno | - | √ | 75.6 | - |
MLEP | IJCAI 19 | 10% test vids with Frame Anno | - | √ | 76.8 | - |
Sultani.etl | ICME 2020 | Weakly (Re-Organized Dataset) | C3D-RGB | X | 86.3 | 0.15 |
IBL | ICME 2020 | Weakly (Re-Organized Dataset) | I3D-RGB | X | 82.5 | 0.10 |
GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | C3D-RGB | √ | 76.44 | - |
GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | TSN-Flow | √ | 84.13 | - |
GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | TSN-RGB | √ | 84.44 | - |
AR-Net | ICME 20 | Weakly (Re-Organized Dataset) | I3D-RGB & I3D Flow | X | 91.24 | 0.10 |
CLAWS | ECCV 20 | Weakly (Re-Organized Dataset) | C3D-RGB | √ | 89.67 | |
MIST | CVPR 21 | Weakly (Re-Organized Dataset) | I3D-RGB | √ | 94.83 | 0.05 |
MTN-KMIL | arxiv 21 | Weakly (Re-Organized Dataset) | I3D-RGB | X | 96.14 | - |
Performance Comparison on Avenue
Model | Reported on Conference/Journal | Supervision | Feature | End2End | AUC(%) |
---|---|---|---|---|---|
Conv-AE | CVPR 16 | Un | - | √ | 70.2 |
Conv-AE* | CVPR 18 | Un | - | √ | 80.0 |
ConvLSTM-AE | ICME 17 | Un | - | √ | 77.0 |
DeepAppearance | ICAIP 17 | Un | - | √ | 84.6 |
Unmasking | ICCV 17 | Un | 3D gradients+VGG conv5 | X | 80.6 |
stacked-RNN | ICCV 17 | Un | - | √ | 81.7 |
FramePred | CVPR 18 | Un | - | √ | 85.1 |
Mem-AE | ICCV 19 | Un | - | √ | 83.3 |
Appearance-Motion Correspondence | ICCV 19 | Un | - | √ | 86.9 |
FramePred* | IJCAI 19 | Un | - | √ | 89.2 |
MNAD | CVPR 20 | Un | - | √ | 88.5 |
VEC | ACM MM 20 | Un | - | √ | 90.2 |
ST-Graph | ACM MM 20 | Un | - | √ | 89.6 |
CAC | ACM MM 20 | Un | - | √ | 87.0 |
AMMC | AAAI 21 | Un | - | √ | 86.6 |
SSMT | CVPR 21 | Un | - | √ | 92.8 |
MLEP | IJCAI 19 | 10% test vids with Video Anno | - | √ | 91.3 |
MLEP | IJCAI 19 | 10% test vids with Frame Anno | - | √ | 92.8 |
Performance Comparison on XD-Violence
Model | Reported on Conference/Journal | Supervision | Feature | Encoder-based | 32 Segments | AP(%) |
---|---|---|---|---|---|---|
Sultani et al. | ECCV 2020 (reported by Wu) | Weakly | I3D-RGB | X | √ | 73.20 |
Wu et al. | ECCV 2020 | Weakly | C3D-RGB | X | X | 67.19 |
Wu et al. | ECCV 2020 | Weakly | I3D-RGB+Audio | X | X | 78.64 |
MTN-KMIL | arxiv 21 | Weakly | I3D-RGB | X | √ | 77.81 |