dangweili / pedestrian-attribute-recognition-pytorch

A simple baseline for pedestrian attribute recognition in surveillance scenarios

Implement of Deep Multi-attribute Recognition model under ResNet50 backbone network

Preparation

Prerequisite: Python 2.7 and Pytorch 0.3.1

1. Install Pytorch

2. Download and prepare the dataset as follow:

   a. PETA Baidu Yun, passwd: 5vep, or Google Drive.

   ./dataset/peta/images/*.png
   ./dataset/peta/PETA.mat
   ./dataset/peta/README
   

   python script/dataset/transform_peta.py 
   

   b. RAP Google Drive.

   ./dataset/rap/RAP_dataset/*.png
   ./dataset/rap/RAP_annotation/RAP_annotation.mat
   

   python script/dataset/transform_rap.py
   

   c. PA100K Links

   ./dataset/pa100k/data/*.png
   ./dataset/pa100k/annotation.mat
   

   python script/dataset/transform_pa100k.py 
   

   d. RAP(v2) Links.

   ./dataset/rap2/RAP_dataset/*.png
   ./dataset/rap2/RAP_annotation/RAP_annotation.mat
   

   python script/dataset/transform_rap2.py
   

Train the model

sh script/experiment/train.sh

Test the model

sh script/experiment/test.sh

Demo

python script/experiment/demo.py

Citation

Please cite this paper in your publications if it helps your research:

@inproceedings{li2015deepmar,
    author = {Dangwei Li and Xiaotang Chen and Kaiqi Huang},
    title = {Multi-attribute Learning for Pedestrian Attribute Recognition in Surveillance Scenarios},
    booktitle = {ACPR},
    pages={111--115},
    year = {2015}
}

Thanks

Partial codes are based on the repository from Houjing Huang.

The code should only be used for academic research.

pedestrian-attribute-recognition-pytorch's Issues

Is it possible to find accuracies of each attribute for both training and testing cases?

I need to find accuracy for all attributes. I have referred to baseline/util/evaluate.py file, it contains 'result' object that has got 'label_acc', 'instance_precision',.etc. How can I extract accuracy of each attribute like 'Age16-30', 'Age31-45', 'Jacket',.etc.
Thank you.

weight problem

weights = torch.zeros(targets_var.shape)
for i in range(targets_var.shape[0]):
for j in range(targets_var.shape[1]):
if targets_var.data.cpu()[i, j] == -1:
weights[i, j] = weight_neg[j]
elif targets_var.data.cpu()[i, j] == 1:

if targets_var.data.cpu()[i, j] == -1: should be changed to targets_var.data.cpu()[i, j] == 0

How can I make a .mat file for my own dataset to train the DeepMAR model?

I see that ./dataset/peta/ contains PETA.mat file. This file contains mapping between images and labels. I also understand that script/dataset/transform_peta.py converts .mat file to .pkl. I need to know how to generate .mat file for my own images.
Thank you.

issue for multiple pedestrians

can it work for multiple pedestrians in a picture? Should I add a human detection before this network?

Hyperparameters used for each dataset?

Hello there! Thank you for sharing this project on Github.

My team is having trouble achieving good performance for RAPv2 and PA100k. We have tried using a range of hyperparameters using what is in ./script/experiment/train.sh as a baseline but we cannot achieve mean accuracy greater than 0.77 for the label based evaluation.

Do you mind sharing what hyperparameters were used for each dataset? We would greatly appreciate it. Thank you!

pretrained mdoel request

This is a request not an issue,
Could you send me the pretrained model you have ?

继续之前的权重训练，出现这个问题，想问下大概什么原因呢？谢谢

Resume from ckpt E:/backup/gitlab/pedestrian-attribute-recognition-pytorch/exp/deepmar_resnet50/joint/partition0/run1/model/ckpt_epoch20.pth,
epoch: 20, scores: 0
Traceback (most recent call last):
File "./script/experiment/train_deepmar_resnet50.py", line 322, in
transfer_optim_state(state=optimizer.state, device_id=0)
File "E:\backup\gitlab\pedestrian-attribute-recognition-pytorch\script\experiment\baseline\utils\utils.py", line 201, in transfer_optim_state
transfer_optim_state(val, device_id=device_id)
File "E:\backup\gitlab\pedestrian-attribute-recognition-pytorch\script\experiment\baseline\utils\utils.py", line 203, in transfer_optim_state
raise RuntimeError("Oops, state[{}] is a Variable!".format(key))
RuntimeError: Oops, state[momentum_buffer] is a Variable!

improve the recognition speed

Hello, after I trained the attacked model, it took a long time to test a picture. I tested a picture on the CPU for 3~4s. Is this normal? How to improve the recognition speed? thank you very much

Whether to support multi GPU for this project?Thank you!

请问可以做成标注功能嘛

具体地说，就是:
标注的行人属性包括性别、头发长度和上下身衣着、鞋子及包的种类和颜色，并提供图像中行人头部、上身、下身、脚、帽子、包位置的标注。

any pretrained model?

Could you please release your pretrained model?

Many thanks.

Add more attributes in PETA dataset

Thank you for your work. I would like to add color attributes into exist 35 attributes, but I do not know how they are indexed. According to https://github.com/valencebond/Rethinking_of_PAR/blob/master/dataset/pedes_attr/annotation.md, I know that their index begin at 35 and end at 79. Did you know any documents have this information?

Can it be applied to video in real time?

Can it be applied to video in real time?

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

what version of cudnn and cuda are you using for this?

Directly resize input image to 224x224 ?

I can see in your paper you first resize images to 256x256 then crop 227xx227: paper

In addition, the images are resized to 256×256 first. After that, they are randomly mirrored and cropped to 227×227 to add the training data

However in your code you directly resize input images to 224x224.

Code snippet from ./script/experiment/train.sh:

python ./script/experiment/train_deepmar_resnet50.py \
    ....
    --resize="(224,224)" \

Did I miss anything? Because I thought random-crop might work better in some classification tasks.

modify and select the attributes I want to train

I want to select some properties in the attribute dataset for training and testing. How do I modify and select the attributes I need to train in train.prototxt and deploy.prototxt? thank you very much

Changes required to run with python3

Hi, What changes are required to run the model in python3. So far, I've changed the cpickle instances to pickle

from import cPickle to pickle to import pickle

and added binary mode while reading and writing i.e. rb.

I've also changed the instances of has_key for dict to __contains__ in DeepMAR.py`.

I'm encountering the following error now:

cfg.__dict__
{'att_list': ['personalLess30',
              'personalLess45',
              'personalLess60',
              'personalLarger60',
              'carryingBackpack',
              'carryingOther',
              'lowerBodyCasual',
              'upperBodyCasual',
              'lowerBodyFormal',
              'upperBodyFormal',
              'accessoryHat',
              'upperBodyJacket',
              'lowerBodyJeans',
              'footwearLeatherShoes',
              'upperBodyLogo',
              'hairLong',
              'personalMale',
              'carryingMessengerBag',
              'accessoryMuffler',
              'accessoryNothing',
              'carryingNothing',
              'upperBodyPlaid',
              'carryingPlasticBags',
              'footwearSandals',
              'footwearShoes',
              'lowerBodyShorts',
              'upperBodyShortSleeve',
              'lowerBodyShortSkirt',
              'footwearSneaker',
              'upperBodyThinStripes',
              'accessorySunglasses',
              'lowerBodyTrousers',
              'upperBodyTshirt',
              'upperBodyOther',
              'upperBodyVNeck'],
 'demo_image': './dataset/demo/demo_image.png',
 'load_model_weight': True,
 'mean': [0.485, 0.456, 0.406],
 'model_kwargs': {'last_conv_stride': 2, 'num_att': 35},
 'model_weight_file': '/nfs/interns/kharshit/pedestrian-attribute-recognition-with-GCN/ckpt_epoch150.pth',
 'rand_seed': None,
 'resize': (224, 224),
 'set_seed': False,
 'std': [0.229, 0.224, 0.225],
 'sys_device_ids': (0,)}
------------------------------------------------------------
Traceback (most recent call last):
  File "script/experiment/demo.py", line 112, in <module>
    model = DeepMAR_ResNet50(**cfg.model_kwargs)
  File "/nfs/interns/kharshit/pedestrian-attribute-recognition-pytorch/baseline/model/DeepMAR.py", line 41, in __init__
    self.base = resnet50(pretrained=self.pretrained, last_conv_stride=self.last_conv_stride)
  File "/nfs/interns/kharshit/pedestrian-attribute-recognition-pytorch/baseline/model/resnet.py", line 189, in resnet50
    model.load_state_dict(remove_fc(model_zoo.load_url(model_urls['resnet50'])))
  File "/nfs/interns/kharshit/pedestrian-attribute-recognition-pytorch/baseline/model/resnet.py", line 151, in remove_fc
    for key, value in state_dict.items():
RuntimeError: OrderedDict mutated during iteration

rapv2 datasets

hi
where is the rapv2 datasets and annotations.zip file，or how can I get the datasets
thank you

The performance of the model on RAP datasets

Have you tested the performance of trained model on RAP dataset? I have trained the first 51 attributes of RAP dataset with ResNet50, but the result is not good enough compared to the paper. the result is as follow:
mA: 0.7549
Acc: 0.5831, Prec: 0.7393, Rec: 0.7097, F1: 0.7242

Pretrained model request

Sorry @dangweili, but could you please to send me your pretrained model with RAPv2 dataset if you have it? I need it for trying to test. Thank you.

the score on the demo_image_result.png

Hello, I chose a picture to test, but I don't understand the score on the demo_image_result.png. What does this score mean? Thank you very much

b、c、d数据集下载不了

你好，由于没有梯子，请问可以把剩余三个，也上传到百度云嘛？感激不尽

activation functions to the final results

Excuse me, have you tried some activation functions(sigmoid, softmax etc.) to the final score of resnet50? can these operations improve the performance of the model? I am now trying.

read issue

(python2-env) D:\codes\pedestrian-attribute-recognition-pytorch-master>python transform_peta.py
Traceback (most recent call last):
File "transform_peta.py", line 83, in
generate_data_description(save_dir)
File "transform_peta.py", line 28, in generate_data_description
data = loadmat(open('./dataset/peta/PETA.mat', 'r'))
File "D:\Anaconda\envs\python2-env\lib\site-packages\scipy\io\matlab\mio.py", line 208, in loadmat
matfile_dict = MR.get_variables(variable_names)
File "D:\Anaconda\envs\python2-env\lib\site-packages\scipy\io\matlab\mio5.py", line 292, in get_variables
res = self.read_var_array(hdr, process)
File "D:\Anaconda\envs\python2-env\lib\site-packages\scipy\io\matlab\mio5.py", line 252, in read_var_array
return self._matrix_reader.array_from_header(header, process)
File "mio5_utils.pyx", line 675, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 723, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 969, in scipy.io.matlab.mio5_utils.VarReader5.read_struct
File "mio5_utils.pyx", line 673, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix
File "mio5_utils.pyx", line 705, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 778, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex
File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric
File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element
File "streams.pyx", line 195, in scipy.io.matlab.streams.ZlibInputStream.read_string
File "streams.pyx", line 171, in scipy.io.matlab.streams.ZlibInputStream.read_into
File "streams.pyx", line 158, in scipy.io.matlab.streams.ZlibInputStream._fill_buffer
zlib.error: Error -3 while decompressing: invalid distance too far back

about test result

Thanks for your excellent work and code!
But i get a issue when testing the model trained by myself using peta dataset, the result is far different from the labels.
Do you have the pretrained model work well that i can do some test?
Thanks for your reply.

How to add my own datasets?

Could you please tell me how to translate the anotations of my own datasets to the correct format in your project, like '.mat'. Thank you.

UnicodeDecodeError

when I tried the "python script/dataset/transform_rap.py", I got the UnicodeDecodeError.

Traceback (most recent call last):
File "script/dataset/transform_rap.py", line 75, in
generate_data_description(save_dir)
File "script/dataset/transform_rap.py", line 28, in generate_data_description
data = loadmat(open('./dataset/rap/RAP_annotation/RAP_annotation.mat', 'r'))
File "/home/cho/anaconda3/envs/visdial-chal/lib/python3.6/site-packages/scipy/io/matlab/mio.py", line 141, in loadmat
MR, file_opened = mat_reader_factory(file_name, appendmat, **kwargs)
File "/home/cho/anaconda3/envs/visdial-chal/lib/python3.6/site-packages/scipy/io/matlab/mio.py", line 65, in mat_reader_factory
mjv, mnv = get_matfile_version(byte_stream)
File "/home/cho/anaconda3/envs/visdial-chal/lib/python3.6/site-packages/scipy/io/matlab/miobase.py", line 222, in get_matfile_version
mopt_bytes = fileobj.read(4)
File "/home/cho/anaconda3/envs/visdial-chal/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xee in position 133: invalid continuation byte

What can I do to resolve this problem?

color enquiry

hello, is the deepMAR model able to predict the colour of clothings ( upper and lower wear)?

chose the gender and upper and lower color attributes of the rap dataset

Hello, I chose the gender and upper and lower color attributes of the rap dataset to train, but the above error occurred. Why is this? Thank you

speed of the model

Thanks for your great work, I trained a model using PETA datasets and tested the 35 pedestrian attributes using PA100k datasets. However, the speed I got is very slow (demo.py), only about 1.2FPS. Is this normal? any suggestions to improve efficiency, thanks!

pa100k_dataset.pkl

哪里可以获得pa100k_dataset.pkl 文件？

output attribute meaning

Hello, thanks for sharing your great work!
Could you help me understand the following sentence in the paper, "the DeepMAR output a label vector which represents whether it has each attribute or not"? For example, in the case of the age prediction, PETA dataset which had four attributes 16-30, 31-45, 45-60 and >60 in your selected 35 attributes, If more than 1 output has a positive score, (eg. first twos) it means the image has both 16-30 and 31-45 attribute? or we need sort above four output scores and vote the biggest positive number as the age prediction? I noticed one more attribute <16 later in the PETA datasets. have you performed some experiments about this? thanks again!

problem in running demo.py

Hello @dangweili ,
I converted py2 files to py3 and am using PyTorch 0.3.1 as specified.
I get a Runtime error (screenshot is attached.)

Can you please guide me if I'm missing something.
Thanks a lot.

你好，吧、

Code run error

I downloaded the file and placed it in the corresponding folder. Here is an error:IOError: [Errno 2] No such file or directory: 'dataset/peta/PETA.mat' i need help.thank you

The test result

Hi, I want to use this model as baseline, and I have trained this model in peta, but the result in test set was so high that I doubt whether my training way was wrong . So can you show your result and training detail? ( I am a new guy in DL)

Why is the number of pedestrian attributes of PETA dataset limited to 35?

PETA dataset includes 61 binary attributes and 4 multi-class attributes, why does the format_peta.py use only 35 attributes? could i use all attributes by changing format_peta.py?

Implement model under Resnet 101 backbone network

Hi,

How to replace resnet50 into resnet101
.

Thks.

Can we train the model with all datasets?

strange error

after I run sh script/experiment/train.sh

Traceback (most recent call last):
File "./script/experiment/train_deepmar_resnet50.py", line 18, in
from baseline.dataset import add_transforms
ImportError: No module named baseline.dataset

baseline.dataset exists, it puzzles me

Is there a demo file?

Hello.
Is there a demo file for testing other image?
I want to know DeepMAR's performance for other person image which is not pedestrian image.