TensorFlow implementation of ASLFeat for CVPR'20 paper "ASLFeat: Learning Local Features of Accurate Shape and Localization", by Zixin Luo, Lei Zhou, Xuyang Bai, Hongkai Chen, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang and Long Quan.
This paper presents a joint learning framework of local feature detectors and descriptors. Two aspects are addressed to learn a powerful feature: 1) shape-awareness of feature points, and 2) the localization accuracy of keypoints. If you find this project useful, please cite:
@article{luo2020aslfeat,
title={ASLFeat: Learning Local Features of Accurate Shape and Localization},
author={Luo, Zixin and Zhou, Lei and Bai, Xuyang and Chen, Hongkai and Zhang, Jiahui and Yao, Yao and Li, Shiwei and Fang, Tian and Quan, Long},
journal={Computer Vision and Pattern Recognition (CVPR)},
year={2020}
}
Please use Python 3.7, install NumPy, OpenCV (3.4.2) and TensorFlow (1.15.2). Refer to requirements.txt for some other dependencies.
If you are using conda, you may configure ASLFeat as:
conda create --name aslfeat python=3.7 -y && \
pip install -r requirements.txt && \
conda activate aslfeat
Clone the repo and download the pretrained model:
git clone https://github.com/lzx551402/aslfeat.git && \
cd ASLFeat/pretrained && \
wget https://research.altizure.com/data/aslfeat_models/aslfeat.tar && \
tar -xvf aslfeat.tar
A quick example for image matching can be called by:
cd /local/aslfeat && python image_matching.py --config configs/matching_eval.yaml
You will be able to see the matching results by displaying disp.jpg.
You may configure configs/matching_eval.yaml
to test images of your own.
1. Benchmark on HPatches dataset
TODO
2. Benchmark on FM-Bench
Download the (customized) evaluation pipeline, and follow the instruction to download the testing data:
git clone https://github.com/lzx551402/FM-Bench.git
Configure configs/fmbench_eval.yaml
and call:
cd /local/aslfeat && python evaluations.py --config configs/fmbench_eval.yaml
The extracted features will be stored in FM-Bench/Features_aslfeat
. Use Matlab to run Pipeline/Pipeline_Demo.m"
then Evaluation/Evaluate.m
to obtain the results.
3. Benchmark on visual localization
Download the Aachen Day-Night dataset and follow the instructions to configure the evaluation.
Configure data_root
in configs/aachen_eval.yaml
, and call:
cd /local/aslfeat && python evaluations.py --config configs/aachen_eval.yaml
The extracted features will be saved alongside their corresponding images, e.g., the features for image /local/Aachen_Day-Night/images/images_upright/db/1000.jpg
will be in the file /local/Aachen_Day-Night/images/image_upright/db/1000.jpg.aslfeat
(the method name here is aslfeat
).
Finally, refer to the evaluation script to generate and submit the results to the challenge website.
4. Benchmark on Oxford Buildings dataset for image retrieval
Take Oxford Buildings dataset as an example. First, download the evaluation data and (parsed) groundtruth files:
mkdir Oxford5k && \
cd Oxford5k && \
mkdir images && \
wget https://www.robots.ox.ac.uk/~vgg/data/oxbuildings/oxbuild_images.tgz && \
tar -xvf oxbuild_images.tgz -C images && \
wget https://research.altizure.com/data/aslfeat_models/oxford5k_gt_files.tar && \
tar -xvf ...
This script also allows for evaluating Paris dataset. The (parsed) groundtruth files can be found here. Be noted to delete the corrupted images of the dataset, and put the remaining images under the same folder.
Next, configure configs/oxford_eval.yaml
, and extract the features by:
cd /local/aslfeat && python evaluations.py --config configs/oxford_eval.yaml
We use Bag-of-Words (BoW) method for image retrieval. To do so, clone and compile libvot:
cd Oxford5k && \
git clone https://github.com/hlzz/libvot.git && \
mkdir build && \
cd build && \
cmake -DLIBVOT_BUILD_TESTS=OFF -DLIBVOT_USE_OPENCV=OFF .. && \
make
and the mAP can be obtained by:
cd Oxford5k && \
python benchmark.py --method_name aslfeat_ms
Please cite libvot if you find it useful.
5. Benchmark on ETH dataset
TODO
6. Benchmark on IMW2020
Download the data (validation/test) Link, then configure configs/imw2020_eval.yaml
, finally call:
cd /local/aslfeat && python evaluations.py --config configs/imw2020_eval.yaml