hzhang57 / awesome-transformer-for-vision

Awesome Transformer for Vision Resources List

A curated list of papers & resources linked to Transformer-based research mainly for vision and graphics tasks.

Contents

• Papers

  • Original Paper
  • 2D Vision Tasks
    • Detection
    • Segmentation
    • Tracking
    • Image Synthesis
    • Action Understanding
  • 3D Vision Tasks
    • Point Cloud Processing
    • Motion Modeling
    • Human Body Modeling
  • Others
    • Music Modeling

• Contributing

Papers

Original Paper

Attention Is All You Need. Ashish Vaswani*, Noam Shazeer*, Niki Parmar*, Jakob Uszkoreit*, Llion Jones*, Aidan N. Gomez*, Łukasz Kaiser*, Illia Polosukhin*. NIPs 2017.

2D Vision Tasks

Detection

Toward Transformer-Based Object Detection. Josh Beal*, Eric Kim*, Eric Tzeng, Dong Huk Park, Andrew Zhai, Dmitry Kislyuk. Arxiv 2020.

Rethinking Transformer-based Set Prediction for Object Detection. Zhiqing Sun*, Shengcao Cao*, Yiming Yang, Kris Kitani. Arxiv 2020.

UP-DETR: Unsupervised Pre-training for Object Detection with Transformers. Zhigang Dai1, Bolun Cai, Yugeng Lin, Junying Chen. Arxiv 2020.

Segmentation

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip H.S. Torr, Li Zhang. Arxiv 2020.

End-to-End Video Instance Segmentation with Transformers. Yuqing Wang, Zhaoliang Xu, Xinlong Wang, Chunhua Shen, Baoshan Cheng, Hao Shen, Huaxia Xia. Arxiv 2020.

Tracking

TransTrack: Multiple-Object Tracking with Transformer. Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Zehuan Yuan, Changhu Wang, Ping Luo. Arxiv 2020.

Image Synthesis

Taming Transformers for High-Resolution Image Synthesis. Patrick Esser*, Robin Rombach*, Bjorn Ommer. Arxiv 2020.

Action Understanding

Video Action Transformer Network. Rohit Girdhar, Joao Carreira, Carl Doersch, Andrew Zisserman. CVPR 2019.

3D Vision Tasks

Point Cloud Processing

PCT: Point Cloud Transformer. Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R. Martin, Shi-Min Hu. Arxiv 2020.

Point Transformer. Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, Vladlen Koltun. Arxiv 2020.

Motion Modeling

Learning to Generate Diverse Dance Motions with Transformer. Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, Hao Li. Arxiv 2020.

A Spatio-temporal Transformer for 3D Human Motion Prediction. Emre Aksan*, Peng Cao*, Manuel Kaufmann, Otmar Hilliges. Arxiv 2020.

Human Body Modeling

End-to-End Human Pose and Mesh Reconstruction with Transformers. Kevin Lin, Lijuan Wang, Zicheng Liu. Arxiv 2020.

Others

Music Modeling

MUSIC TRANSFORMER: GENERATING MUSIC WITH LONG-TERM STRUCTURE. Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck. Arxiv 2018.

Contributing

Please see CONTRIBUTING for details.