yangmingbj / diml

[ICCV 2021] Towards Interpretable Deep Metric Learning with Structural Matching

DIML

Created by Wenliang Zhao*, Yongming Rao*, Ziyi Wang, Jiwen Lu, Jie Zhou

This repository contains PyTorch implementation for paper Towards Interpretable Deep Metric Learning with Structural Matching (ICCV 2021).

We present a deep interpretable metric learning (DIML) that adopts a structural matching strategy to explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. Our method enables deep models to learn metrics in a more human-friendly way, where the similarity of two images can be decomposed to several part-wise similarities and their contributions to the overall similarity. Our method is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods.

[arXiv]

Usage

Requirement

• python3
• PyTorch 1.7

Dataset Preparation

Please follow the instruction in RevisitDML to download the datasets and put all the datasets in data folder. The structure should be:

data
├── cars196
│   └── images
├── cub200
│   └── images
└── online_products
    ├── images
    └── Info_Files

Training & Evaluation

To train the baseline models, run the scripts in scripts/baselines. For example:

CUDA_VISIBLE_DEVICES=0 ./script/baselines/cub_runs.sh

The checkpoints are saved in Training_Results folder.

To test the baseline models with our proposed DIML, first edit the checkpoint paths in test_diml.py, then run

CUDA_VISIBLE_DEVICES=0 ./scripts/diml/test_diml.sh cub200

The results will be written to test_results/test_diml_<dataset>.csv in CSV format.

You can also incorporate DIML into the training objectives. We provide two examples which apply DIML to Margin and Multi-Similarity loss. To train DIML models, run

# ./scripts/diml/train_diml.sh <dataset> <batch_size> <loss> <num_epochs>
# where loss could be margin_diml or multisimilarity_diml
# e.g.
CUDA_VISIBLE_DEVICES=0 ./scripts/diml/train_diml.sh cub200 112 margin_diml 150

Acknowledgement

The code is based on RevisitDML.

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{zhao2021towards,
  title={Towards Interpretable Deep Metric Learning with Structural Matching},
  author={Zhao, Wenliang and Rao, Yongming and Wang, Ziyi and Lu, Jiwen and Zhou, Jie},
  booktitle={ICCV},
  year={2021}
}