willard-yuan / awesome-cbir-papers

📝Awesome and classical image retrieval papers

CBIR in academia and industry

Awesome image retrieval papers

The main goal is to collect classical and solid works of image retrieval in academia and industry.

• Classical Local Feature
• Deep Learning Feature (Global Feature)
• Deep Learning Feature (Local Feature)
• Deep Learning Feature (Instance Search)
• ANN search
• CBIR Attack
• CBIR rank
• CBIR in Industry
• CBIR Competition and Challenge
• CBIR for Duplicate(copy) detection
• Feature Fusion
• Instance Matching
• Semantic Matching
• Template Matching
• Image Identification
• Tutorials
• Slide
• Demo and Demo Online
• Datasets
• Useful Package

Classical Local Feature

• Object retrieval with large vocabularies and fast spatial matching, CVPR 2007.
• Visual Categorization with Bags of Keypoints, ECCV 2004.
• ORB: an efficient alternative to SIFT or SURF, ICCV 2011.
• Object Recognition from Local Scale-Invariant Features, ICCV 1999.
• Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval, ICCV 2007.
• Three things everyone should know to improve object retrieval, CVPR 2012.
• On-the-fly learning for visual search of large-scale image and video datasets
• All about VLAD, CVPR 2013.
• Aggregating localdescriptors into a compact image representation, CVPR 2010.
• More About VLAD: A Leap from Euclidean to Riemannian Manifolds, CVPR 2015.
• Hamming embedding and weak geometric consistency for large scale image search, CVPR 2008.
• Revisiting the VLAD image representation, project
• Improving the Fisher Kernel for Large-Scale Image Classification, ECCV 2010.
• Image Classification with the Fisher Vector: Theory and Practice
• Democratic Diffusion Aggregation for ImageRetrieval
• A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval, ACCV 2016.
• Triangulation embedding and democratic aggregation for image search, CVPR 2014.
• Efficient Large-scale Image Search With a Vocabulary Tree, IPOL 2015, code.

Deep Learning Feature (Global Feature)

• Online Invariance Selection for Local Feature Descriptors, ECCV 2020, code.
• Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval, ECCV 2020.
• SOLAR: Second-Order Loss and Attention for Image Retrieval, ECCV 2020.
• Unifying Deep Local and Global Features for Image Search, arxiv 2020.
• SOLAR: Second-Order Loss and Attention for Image Retrieval, arxiv 2020.
• A Benchmark on Tricks for Large-scale Image Retrieval，arxiv 2020.
• Learning with Average Precision: Training Image Retrieval with a Listwise Loss, ICCV 2019.
• MultiGrain: a unified image embedding for classes and instances, arxiv 2019.
• Deep Image Retrieval:Learning Global Representations for Image search.
• End-to-end Learning of Deep Visual Representations for Image retrieval, DIR更详细的论文说明.
• What Is the Best Practice for CNNs Applied to Visual Instance Retrieval?, 关于layer选取的问题.
• Bags of Local Convolutional Features for Scalable Instance Search.
• Faster R-CNN Features for Instance Search, CVPR workshop 2016.
• Cross-dimensional Weighting for Aggregated Deep Convolutional Features, project.
• Class-Weighted Convolutional Features for Image Retrieval.
• Multi-Scale Orderless Pooling of Deep Convolutional Activation Features, VLAD coding.
• Aggregating Deep Convolutional Features for Image Retrieval, 论文笔记, 基于深度学习的视觉实例搜索研究进展.
• Particular object retrieval with integral max-pooling of CNN activations, project.
• Particular object retrieval using CNN.
• Learning to Match Aerial Images with Deep Attentive Architectures.
• Siamese Network of Deep Fisher-Vector Descriptors for Image Retrieval.
• Combining Fisher Vector and Convolutional Neural Networks for Image Retrieval, fv和cnn特征融合提升.
• Selective Deep Convolutional Features for Image Retrieval, ACM MM 2017.
• Class-Weighted Convolutional Features for Image Retrieval.
• Fine-tuning CNN Image Retrieval with No Human Annotation, TPAMI 2018.
• An accurate retrieval through R-MAC+ descriptors for landmark recognition.
• Regional Attention Based Deep Feature for Image Retrieval, code, BMVC 2018.
• Detect-to-Retrieve: Efficient Regional Aggregation for Image Search, CVPR 2019.
• Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking, project, CVPR 2018.
• Guided Similarity Separation for Image Retrieval, NeurIPS 2019.

Deep Learning Feature (Local Feature)

• Glue Factory is CVG's library for training and evaluating deep neural network that extract and match local visual feature, code
• DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching, arXiv 2023, code.
• LightGlue: Local Feature Matching at Light Speed, arXiv 2023, code.
• Simple Learned Keypoints, a self-supervised deep learning keypoint model, arxiv 2023, code.
• Learning Super-Features for Image Retrieval, ICLR 2022, code.
• LoFTR: Detector-Free Local Feature Matching with Transformers, CVPR 2021, code.
• DFM: A Performance Baseline for Deep Feature Matching, CVPRW 2021, code.
• COTR: Correspondence Transformer for Matching Across Images, arxiv 2021.
• Online Invariance Selection for Local Feature Descriptors, ECCV 2020, code.
• Learning and aggregating deep local descriptors for instance-level recognition, ECCV 2020, code.
• DISK: Learning local features with policy gradient, NeurIPS 2020, code.
• Learning and aggregating deep local descriptorsfor instance-level recognition, ECCV 2020, code.
• D2D: Keypoint Extraction with Describe to Detect Approach, arxiv 2020.
• UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision, arxiv.
• Visualizing Deep Similarity Networks, WACV 2019.
• Combination of Multiple Global Descriptors for Image Retrieval.
• Beyond Cartesian Representations for Local Descriptors, code, ICCV 2019.
• R2D2: Reliable and Repeatable Detector and Descriptor, code, NeurIPS 2019.
• SOSNet: Second Order Similarity Regularization for Local Descriptor Learning, CVPR 2019.
• Local Features and Visual Words Emerge in Activations, CVPR 2019.
• Explicit Spatial Encoding for Deep Local Descriptors, CVPR 2019.
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters, ICCV 2019.
• Learning Discriminative Affine Regions via Discriminability, affnet.
• A Large Dataset for Improving Patch Matching, PS-Dataset.
• Working hard to know your neighbor's margins: Local descriptor learning loss, code.
• MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching, code.
• LF-Net: Learning Local Features from Images, NeurIPS 2018.
• Local Descriptors Optimized for Average Precision, CVPR 2018.
• SuperPoint: Self-Supervised Interest Point Detection and Description, Magic Leap.
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints, code, ECCV 2018.
• Learning local feature descriptors with triplets and shallow convolutional neural networks, BMVC 2016.

Deep Learning Feature (Instance Search)

• Deeply Activated Salient Region for Instance Search, arXiv 2020.
• Instance search based on weakly supervised feature learning, Neurocomputing 2019.
• Instance Search via Instance Level Segmentation and Feature Representation, arXiv 2018.
• Unsupervised object discovery for instance recognition, WACV 2018.
• Faster R-CNN Features for Instance Search, CVPR workshop 2016.

ANN search

• Results of the NeurIPS’21 Challenge on Billion-Scale Approximate Nearest Neighbor Search.
• Nearest neighbor search with compact codes: A decoder perspective, arxiv 2021.
• Accelerating Large-Scale Inference with Anisotropic Vector Quantization, blog, code, ICML 2020.
• Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination, SIGMOD 2020.
• RobustiQ A Robust ANN Search Method for Billion-scale Similarity Search on GPUs, ICMR 2019.
• Zoom: Multi-View Vector Search for Optimizing Accuracy, Latency and Memory.
• Vector and Line Quantization for Billion-scale Similarity Search on GPUs.
• GGNN: Graph-based GPU Nearest Neighbor Search, arxiv 2019, code.
• Learning to Route in Similarity Graphs, ICML 2019.
• Practical and Optimal LSH for Angular Distance.
• pq-fast-scan.
• faiss. A library for efficient similarity search and clustering of dense vectors.
• Polysemous codes.
• Optimized Product Quantization.
• lopq. Training of Locally Optimized Product Quantization (LOPQ) models for approximate nearest neighbor search of high dimensional data in Python and Spark.
• nns_benchmark. Benchmark of Nearest Neighbor Search on High Dimensional Data.
• Optimized Product Quantization.
• Falconn. FAst Lookups of Cosine and Other Nearest Neighbors.
• Annoy. Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk.
• NMSLIB. Non-Metric Space Library (NMSLIB): A similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs, graph-based method.
• Fast Approximate Nearest Neighbor Search With Navigating Spreading-out Graphs, code
• Efficient Nearest Neighbors Search for Large-Scale Landmark Recognition
• NV-tree: A Scalable Disk-Based High-Dimensional Index.
• Dynamicity and Durability in Scalable Visual Instance Search.
• Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighbors，code.
• Link and code: Fast indexing with graphs and compact regression codes.
• A Survey of Product Quantization，对于矢量量化方法一篇比较完整的调研，值得一读.
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints，学习局部特征的descriptor，匹配能力较强.
• Learning a Complete Image Indexing Pipeline, CVPR 2018.
• spreading vectors for similarity search, ICLR 2019.
• SPTAG: A library for fast approximate nearest neighbor search. Microsoft.

CBIR Attack

• Open Set Adversarial Examples.

CBIR rank

• Fast Spectral Ranking for Similarity Search, code, CVPR 2018.

CBIR in Industry

• Videntifier is a visual search engine based on a patented large-scale local feature database, demo, based on SIFT feature and NV-tree. (Chinese blog post).
• Web-Scale Responsive Visual Search at Bing.
• Visual Search at Alibaba.
• Visual Search at Pinterest.
• Visual Discovery at Pinterest.
• Learning a Unified Embedding for Visual Search at Pinterest, KDD 2019.
• Visual Search at ebay.
• Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce, project.
• 微信「扫一扫识物」 的背后技术揭秘.
• 揭秘微信「扫一扫」识物为什么这么快？

CBIR Competition and Challenge

• The 2021 Image Similarity Dataset and Challenge, 2021, code.
• Google Landmark Retrieval Challenge, 2018.
• Alibaba Large-scale Image Search Challenge, 2015.
• Pkbigdata image retrieval, 2015.
• Large-scale Landmark Retrieval/Recognition under a Noisy and Diverse Dataset, Landmark2019-1st-and-3rd-Place-Solution.

CBIR for Duplicate(copy) detection

• A Self-Supervised Descriptor for Image Copy Detection, CVPR 2022, code.
• A Robust and Fast Video Copy Detection System Using Content-Based Fingerprinting.

Feature Fusion

• Feature fusion using Canonical Correlation Analysis.

Instance Matching

• Neural- Guided RANSAC: Learning Where to Sample Model Hypotheses, ICCV 2019, code.
• AdaLAM: Revisiting Handcrafted Outlier Detection, arxiv 2006.
• Graph-Cut RANSAC, code
• Image Matching Benchmark
• GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence
• A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval
• CODE: Coherence Based Decision Boundaries for Feature Correspondence
• Robust feature matching in 2.3µs
• PopSift is an implementation of the SIFT algorithm in CUDA
• openMVG robust_estimation
• Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses.
• Homography from two orientation- and scale-covariant features, code.

Semantic Matching

• End-to-end weakly-supervised semantic alignment.

Template Matching

• QATM: Quality-Aware Template Matching For Deep Learning, CVPR 2019.

Image Identification

• Image Identification Using SIFT Algorithm: Performance Analysis against Different Image Deformations.

Tutorials

• PyRetri, Open source deep learning based image retrieval toolbox based on PyTorch.
• How to Apply Distance Metric Learning to Street-to-Shop Problem.
• Recent Image Search Techniques.
• Compact Features for Visual Search.
• multimedia-indexing. A framework for large-scale feature extraction, indexing and retrieval.
• Image Similarity using Deep Ranking, code.
• Triplet Loss and Online Triplet Mining in TensorFlow.
• tf_retrieval_baseline.

Slide

• VRG Prague in “Large-Scale Landmark Recognition Challenge”, ranked 3rd in the Google Landmark Recognition Challenge.

Demo and Demo Online

• Visual Image Retrieval and Localization, SIFT feature encoded by BOW.
• VGG Image Search Engine, SIFT feature encoded by BOW.
• SoTu, A flask-based cbir system.
• yisou, A flask-based painting cbir system, the search algorithm is designed by Yong Yuan.

Datasets

• DeepFashion2 Dataset, DeepFashion2 is a comprehensive fashion dataset.
• Holidays, Holidays consists images from personal holiday albums of various scene types.
• Oxford, Oxford consists of 11 different Oxford landmarks.
• Paris, Paris consists of images crawled from 11 queries on specific Paris architecture.
• ROxford and RParis, ROxford and RParis are revisited versions of the original Oxford and Paris with annotation corrections, enlarged sizes and more difficult samples.
• INSTRE, INSTRE is an instance-level object retrieval dataset.

Useful Package

• VLFeat
• Yael

Star History

awesome-cbir-papers's Issues

一些技术疑问

这个对图像做cnn 特征抽取，特征维度都在几千，有点太了吧，是否能够减少维度 提高效率。还是在搜索的时候用什么算法来做进一步处理

inverted-multi index and hardware accelerated

thanks for this collection,I have watching this topic for a long time. In practice，inverted-multi and PQ combined index saved a lot of search time and space in ANN searching.and some hardware accelarted method proposaled,multi-ivf
fpga and opencl based acc.Efficient Large-Scale Approximate Nearest Neighbor Search on OpenCL FPGA.
I think the hardware combined to ANNS is a trend ,do you think so.

The url to "Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking" is broken

Hi~ I'm also interested in the paper of "Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking", but I did not find it online. I wonder if you could provide another URL for this paper. Thanks!

Significance Test for Visual Search Task

Hi @willard-yuan , have you found some paper recommendations regarding the significance test of our model for visual search tasks? I'm currently working on this task and would like to know if there is any technique to test the significance of the model experiment. I know one significance test, which is Mcnemar's test, but this test is commonly used for classification tasks. I wondered is there a way to use Mcnemar's test in visual search task?

Data request for Alibaba Large-scale Image Search Challenge

你好，

感谢整理这份资料！
阿里的Alibaba Large-scale Image Search Challenge这个竞赛数据现在原始链接已不能下载，请问你那里有保存这个数据集吗？可否上传一份至百度云/Google drive？谢谢！

Thanks for organizing all these materials!
The original download link for Alibaba Large-scale Image Search Challenge data is broken. Do you have a local copy of it? If you do, could you share it with BaiduYunPan / Google drive? Thanks!

some new papers and method suggest to add

9.RobustiQ A Robust ANN Search Method for Billion-scale Similarity Search on GPUs
11.Zoom: Multi-View Vector Search for Optimizing Accuracy, Latency and Memory
12. Vector and Line Quantization for Billion-scale Similarity Search on GPUs
I just found that the GGNN(near exactly search) already added.
Did you done some experiments in the large scale image classification vs deep-metric ? Or some work like search engine guided similarity learning .eg Search Engine Guided Neural Machine Translation?

SIGMOD 2020 Approximate Nearest Neighbor Search Paper

Hi, consider adding the SIGMOD 2020 paper: Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination, https://www.pdl.cmu.edu/PDL-FTP/BigLearning/mod0246-liA.pdf

CVPR 2020 Accepted Image Retrieval Paper

Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval, https://arxiv.org/abs/2002.10310

a new distributed anns library named 'SPTAG' opensource by Microsoft seems very strong.I suggest add.

SPTAG github urlhttps://github.com/microsoft/SPTAG