A curated list of research papers in Vision-and-Language Navigation (VLN). Link to the code and website if available is also present.
Please feel free to contact me via email ([email protected]) or open an issue or submit a pull request.
To add a new paper via pull request:
- Fork the repo, edit
README.md
. - Put the new paper at the correct chronological position as the following format:
1. **Paper Title** <br> *Author(s)* <br> Conference, Year. [[Paper]](link) [[Code]](link) [[Website]](link)
- Send a pull request. Ideally, I will review the request within a week.
-
Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, Anton van den Hengel
CVPR, 2018. [Paper] [Code] [Website] -
Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
Xin Wang, Wenhan Xiong, Hongmin Wang, William Yang Wang
ECCV, 2018. [Paper] -
Speaker-Follower Models for Vision-and-Language Navigation
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, Trevor Darrell
NeurIPS, 2018. [Paper] [Code] [Website] -
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
Jesse Thomason, Daniel Gordon, Yonatan Bisk
NAACL, 2019. [Paper] [Poster] -
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, Lei Zhang
CVPR, 2019. [Paper] -
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, Caiming Xiong
ICLR, 2019. [Paper] [Code] [Website] -
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
Chih-Yao Ma, Zuxuan Wu, Ghassan AlRegib, Caiming Xiong, Zsolt Kira
CVPR, 2019. [Paper] [Code] [Website] -
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, Siddhartha Srinivasa
CVPR, 2019. [Paper] [Code] [Video] -
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Hao Tan, Licheng Yu, Mohit Bansal
NAACL, 2019. [Paper] [Code] -
Multi-modal Discriminative Model for Vision-and-Language Navigation
Haoshuo Huang, Vihan Jain, Harsh Mehta, Jason Baldridge, Eugene Ie
NAACL Workshop, 2019. [Paper] -
Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, Kate Saenko
ACL, 2019. [Paper] -
Chasing Ghosts: Instruction Following as Bayesian State Tracking
Peter Anderson, Ayush Shrivastava, Devi Parikh, Dhruv Batra, Stefan Lee
NeurIPS, 2019. [Paper] -
Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters
Federico Landi, Lorenzo Baraldi, Massimiliano Corsini, Rita Cucchiara
BMVC, 2019. [Paper] [Code] -
Transferable Representation Learning in Vision-and-Language Navigation
Haoshuo Huang, Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Magalhaes, Jason Baldridge, Eugene Ie
ICCV, 2019. [Paper] -
Robust Navigation with Language Pretraining and Stochastic Sampling
Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi
EMNLP, 2019. [Paper][Code] -
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
Fengda Zhu, Yi Zhu, Xiaojun Chang, Xiaodan Liang
arXiv:1911.07883. [Paper] -
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling
Tsu-Jui Fu, Xin Wang, Matthew Peterson, Scott Grafton, Miguel Eckstein, William Yang Wang
arXiv:1911.07308. [Paper]