simon4yan / learning-via-translation

SPGAN in CVPR'18

Learning-via-Translation

Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification (https://arxiv.org/pdf/1711.07027.pdf)

---

Framework Overview

Learning via translation for domain adaptation in person re-ID consists of two steps:

1. Source-target image translation

The first step is to translate the annotated dataset from source domain to target domain in an unsupervised manner.

For more reference, you can find our modified training code and generating code in ./SPGAN. We wrote a detailed README. If you still has some question, feel free to contact me ([email protected]).

2. Feature learning

With the translated dataset that contains labels, feature learning methods are applied to train re-ID models.

Feature Learning

The code for re-ID feature learning is mainly modified from IDE.

Train

1. Train baselines:

 cd $IDE_ROOT
  # train IDE on Market-1501
  ./experiments/market/train_IDE_ResNet_50_baseline.sh  
  # train IDE on DukeMTMC-reID
  ./experiments/duke/train_IDE_ResNet_50_baseline.sh
  # The IDE models for Market-1501 and DukeMTMC-reID are saved under: "out/market_train" and "out/duke_train", respectively
  # If you encounter this problem: bash: ./experiments/market/train_IDE_ResNet_50_baseline.sh: Permission denied
  # Please execute: chmod 777 -R experiments/

2. Train IDE models trained on the translated images:

 cd $IDE_ROOT
  # train IDE on Market-1501
  ./experiments/market/train_IDE_ResNet_50_domain.sh  
  # train IDE on DukeMTMC-reID
  ./experiments/duke/train_IDE_ResNet_50_domain.sh

Feature Extraction

# 1. Market-1501:
cd $IDE_ROOT/market_evaluation
Run Matlab: extract_feature.m
# The IDE features are saved under: "market_evaluation/feat"
# 2. DukeMTMC-reID:
cd $IDE_ROOT/duke_evaluation
Run Matlab: extract_feature.m
# The IDE features are saved under: "duke_evaluation/feat"

Evaluation

  Run Matlab: baseline_evaluation_IDE.m

Trained Model

• Baseline models

  IDE_Market and IDE_Duke: Onedrive; Baidu Yun

• IDE models trained on the translated images:

  IDE_Market2duke (SPGAN) and IDE_Duke2Market (SPGAN): Onedrive; Baidu Yun

Results

(see more details in paper)

Contributing

If you have an implementation for a different framework, or an enhanced version of the current code, feel free to contribute to the repository. For any issues related to the code you can use the issue tracker from GitHub.

References

[1] W. Deng, L. Zheng, Q. Ye, G. Kang, Y. Yang, and J. Jiao. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. arXiv preprint arXiv:1711.07027, 2017.

[2] L. Zheng, Y. Yang, and A. G. Hauptmann. Person reidentification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016.

@inproceedings{image-image18,
 author    = {Weijian Deng and
              Liang Zheng and
              Qixiang Ye and
              Guoliang Kang and
              Yi Yang and
              Jianbin Jiao},
 title     = {Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity
              for Person Re-identification},
 booktitle = {CVPR},
 year      = {2018},
}

License

The code is released under the MIT License. See the attached LICENSE file.

Frequently Asked Questions

Q1: The direct transfer performance using pytorch is much lower than caffe.

Ans: We find the BN is the key factor to this gap, and see more in here.

Q2: Train IDE or IDE+ by using pytorch?

Ans: Yes, we recommend Person_reID_baseline_pytorch. Note that bn needed to be fixed (see Q1).

(We also give a simple code of IDE and IDE+ here, which is modifed from Person_reID_baseline_pytorch).

Q3: The pytorch vision of "Learning via translation".

Ans: Yes, please see here.

learning-via-translation's Issues

the computation of contrastive loss

In "Learning-via-Translation\SPGAN\models\models_spgan.py", why the computation of contrastive loss divide the number of "3.0", like this "m_loss = tf.identity(( T_metric_POS + S_metric_POS + 2*NEG)/3.0 , name = 'metric_losses')", is that should be "4.0"?

what's the meaning of numClass?

sorry,i don't understand the meaning of numClass,which in the LOMO_XQDA/code/Demo_XQDA.m

numClass = 632;
numFolds = 10;
numRanks = 100;

%% load the extracted LOMO features

test error when load net

when test use your model, this line "net = caffe.Net(model, weights, 'test');" cause error:

run extract_feature.m
*** Error in `/MATLABR2014a/bin/glnxa64/MATLAB': double free or corruption (out): 0x00007fac72ae6560 ***

The images generated by Cycle-GAN

Hi,I use Cycle-GAN to generate images from Market to Duke.But the results are bad.I find the results in your paper are nice.Could you tell me some parameter settings e.g. epoch,learning rate,image size ...

The result is lower than this paper

when I training on market and testing on duke, the rank-1 and mAP are around 35%,18%.when I training on duke and testing on market, the result is as same as this paper. If there are some different set when testing on duke? The direct transfer rank-1 is 30% when testing on duke, it's also lower than paper..

Where is the LMP in code?

Hello, I would like to learn about Local Max Pooling (LMP), but I can't find it in all the files. Could you please tell me where it is? Thanks!

Difference between ResNet_50 and ResNet_50_domain model

Why do you create two difference models for ResNet_50 and ResNet_50_domain? What's the difference and why? Thanks for your help.

Trained model and translated images of query set and gallery set

Hi, can you provide the trained well model that maps images of market-1501 to the duke style? or the query and gallery images of market-1501 that are mapped to duke style?

how much size of target dataset is needed?

Hi,

I wanna try transfer market1501 to my own dataset

I wonder that have you ever tried how much size of target dataset is needed for

getting good transfer result for SPGAN??

The download link of dataset is not available

The following download links is not available. Could you share the new download links? Thank you very much.

Market-1501: http://liangzheng.com.cn/Project/project_reid.html
DukeMTMC-reID: https://github.com/layumi/DukeMTMC-reID_evaluation

How to implement feature learning?

Hi, author! Thank for your perfect work.I have met some problems with feature learning this part.
cd $IDE_ROOT

train IDE on Market-1501

./experiments/market/train_IDE_ResNet_50_baseline.sh
I execute this command，and the error is : caffe /build/.../caffe : not a dictionary.
So how can I make this work? May I execute IDE with pytorch? Waiting for your reply. Thank you!

ImageNet pretrained caffe model

When we try to train the baseline using train_IDE_ResNET_50_baseline.sh we get an error saying that this directory 'data/imagenet_models//ResNet_50.caffemodel' was not found. We only found a readme file in 'imagenet_models'. We were wondering if we are supposed to add an ImageNet pretrained ResNet caffe model in this directory? If so, could you provide a link or tell us where to find it?

Thanks!

new questions

Hello author, I'm sorry to disturb your rest,
What I want to inquire about is whether the project you published can transfer the CHUK or Market dataset to the style of the COCO dataset? I hope to receive your reply
Thank you for your contribution to this article. It's really great!

Direct transfer is more than style conversion accuracy

Environment: Pytorch 0.4 Python 3.5 Titan XP
Problem: I trained directly on the dukemtmc dataset and tested it on the market1501 dataset. MAP:40%, Top1: 66.6%.I trained on the duk2market dataset and tested it on the market1501 dataset. MAP:22.6%, Top1: 50.1%.I downloaded duke2market on your side.This is somewhat inconsistent with the description of the paper.The reid model is referenced https://github.com/dapengchen123/crf_affinity.Can you help me?Thank you.

SPGAN training problem

follow the instructions，I train the SPGAN， but Every time I get stuck in training for the first epoch。
#######
2018-11-07 21:52:58.269039: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
[] Loading checkpoint...
None
[] No suitable checkpoint!
================ Begining Training time (Wed Nov 7 21:54:31 2018) ================

---

Epoch: ( 0) ( 1/ 50)
######################

It was repeated many times, once to the second epoch，But it's stuck again

################

2018-11-07 21:52:58.268951: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-11-07 21:52:58.269018: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-11-07 21:52:58.269039: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
[] Loading checkpoint...
None
[] No suitable checkpoint!
================ Begining Training time (Wed Nov 7 21:54:31 2018) ================

---

Epoch: ( 0) ( 1/ 50)

Epoch: ( 0) ( 6/ 50)

Epoch: ( 0) ( 11/ 50)

Epoch: ( 0) ( 16/ 50)

Epoch: ( 0) ( 21/ 50)

Epoch: ( 0) ( 26/ 50)

Epoch: ( 0) ( 31/ 50)

Epoch: ( 0) ( 36/ 50)

Epoch: ( 0) ( 41/ 50)

Epoch: ( 0) ( 46/ 50)
================ Training time (Wed Nov 7 22:35:27 2018) ================

Model saved in file: ./checkpoints/market2duke_spgan/Epoch_(0)_(50of50).ckpt

Epoch: ( 1) ( 1/ 50)

################

Anybody have the same problem？

Which chechpoint should I choose in style stransfer phase?

Thank you for your excellent work.
During the style transfer, which epoch checkpoint should I choose? It seems that no difference can be seen between the generated image on different epochs.

How to implement feature learning ,

ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Hello, which version of tensorflow do you use? I installed the version 1.0,but when I run ' python train_spgan_demo.py --dataset=market2duke --gpu_id=2',I got the error:ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory.
How can I solve it ???

the details :
Traceback (most recent call last):
File "train_spgan_demo.py", line 5, in
import Utils.ops as ops
File "/home/xiaozhenzhen/data/Learning-via-Translation/SPGAN/Utils/ops.py", line 6, in
import tensorflow as tf
File "/home/xiaozhenzhen/anaconda2/envs/tf_spgan/lib/python2.7/site-packages/tensorflow/init.py", line 24, in
from tensorflow.python import *
File "/home/xiaozhenzhen/anaconda2/envs/tf_spgan/lib/python2.7/site-packages/tensorflow/python/init.py", line 72, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/home/xiaozhenzhen/anaconda2/envs/tf_spgan/lib/python2.7/site-packages/tensorflow/python/init.py", line 61, in
from tensorflow.python import pywrap_tensorflow
File "/home/xiaozhenzhen/anaconda2/envs/tf_spgan/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in
_pywrap_tensorflow = swig_import_helper()
File "/home/xiaozhenzhen/anaconda2/envs/tf_spgan/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#import_error

Negative training pairs different from the article

In the paper, authors describe that the translated image should be of different id of target image. I think that the neg pairs should be a2b/b, and b2a/a. But in the codes, i find a/b is used for neg pair.

  neg = tf.pow(tf.subtract(a_metric, b_metric), 2)
  neg = tf.reduce_sum(neg, 1)
  neg = tf.sqrt(neg + 1e-6)
  NEG = tf.pow(tf.maximum(tf.subtract(C, neg), 0), 2)

Direct Transfer results

 Hi, I have some questions about your 'Direct Transfer' results in Table 2 in your paper. I make the setting consistent with yours, but I can't get so-high baselines, like rank1 33.1%, mAP 16.7  when train on market1501 and test on Duke-MTMC-reID. Actually I always got this result around rank1 27%, mAP 13%. And this result is better than those in the [https://arxiv.org/pdf/1705.10444.pdf](url), where this result is that rank1 21.9%, mAP 10.9%. 
 I wonder that is there some tricks used in your experiments, and I'm looking forward to your reply .

the code to do multiple-query person reid

Hi, I notice in your paper, you have calculated the multiple-query settings. But I can't find the code.

No test folder in duke_evaluation

In "evaluation_res_duke_fast.m"

there are a line : Hist_query = importdata(['test/IDE_' netname '_query.mat']);

However, I cannot find folder test

Thx~