pangwong / pytorch-multi-label-classifier

A pytorch implemented classifier for Multiple-Label classification

pytorch-multi-label-classifier

Introdution

A pytorch implemented classifier for Multiple-Label classification. You can easily train, test your multi-label classification model and visualize the training process.
Below is an example visualizing the training of one-label classifier. If you have more than one attributes, no doubt that all the loss and accuracy curves of each attribute will show on web browser orderly.

Loss	Accuracy

Module

• data

  data preparation module consisting of reading and transforming data. All data store in data.txtand label.txt with some predefined format explained below.

• model

  scripts to build multi-label classifier model. Your model templets should put here.

• options

  train test and visualization options define here

• util

  • webvisualizer: a visdom based visualization tool for visualizing loss and accuracy of each attribute
  • util: miscellaneous functions used in project
  • html: used in webvisualizer.

• test

  • mnist: mnist dataset arranged as defined data format.
  • celeba: exactract some of attributes of CelebA dataset

Multi-Label Data Format

Data Format Explanation.

• label.txt

 Store attribute information including its name and value. label.txt example. Lines in label.txt stack as follows:

• For each attribute :
  • number of attribute values ; id of attribute ; name attribute
  • For each attribute value belonging to current attribute :
    • id of attibute_value ; name of attribute value

 Note: mind the difference between attribute and attribute value.

• data.txt

 Store objects information including attribute id and bounding box and so on. Each line is one json dict recording one object. data.txt example  

• "box":object boundingbox. 'x': top_left.x , 'y':top_left.y, 'w': width of box, 'h': height of box.
• "image_id": image identifier. An image content dependent hash value.
• "box_id": object identidier. Combine image_id, box['x'], box['y'], box['w'], box["h"] with _.
• "size": image width and height. Used for varifying whether box is valid.
• "id": list of ids. Store multi-label attributes ids, the order is the same as the attributes' order in label.txt

Dependence

• Visdom 0.1.7.2
• Pytorch 0.3.1.post2

TODO

• Snapshot loss and accuracy records
• Support visualize multi top K accuracy
• Support model finetuning
• Complete test module
• Add switch to control loss and accuracy curves displaying on one plot or multiple
• Train and Test Log

Reference

Part of codes and models refer to some other OSS listed below for thanks:

• pytorch-CycleGAN-and-pix2pix
• pytorch-LightCNN

pytorch-multi-label-classifier's Issues

Data not tensor error

Traceback (most recent call last):
File "multi_label_classifier.py", line 337, in
main()
File "multi_label_classifier.py", line 330, in main
train(model, criterion, train_set, val_set, opt, (rid2name, id2rid))
File "multi_label_classifier.py", line 170, in train
output, loss, loss_list = forward_batch(model, criterion, inputs, targets, opt, "Train")
File "multi_label_classifier.py", line 57, in forward_batch
target_vars.append(Variable(targets[index]))
TypeError: Variable data has to be a tensor, but got tuple

I received the above error. Can you please help me understand it. I used label.txt and data.txt in the format given and my dataset is PNGs of 1024x1024. I am trying to train with Resnet50 with load size of 256 and input size of 224. My labels are 8 binary variables with unique names e.g. no_one, yes_one, ... , no_eight, yes_eight. Each label is some combination of the eight attribute values e.g. ["no_one","yes_two",...,"no_seven","no_eight"].

What is the input and target for the loss function?

Say I have 5 classes, what would be the input and target?

argument

hi,i run the multi_label_classifier.py and met the following error:
usage: multi_label_classifier.py [-h] --dir DIR --name NAME --mode MODE
--model MODEL [--load_size LOAD_SIZE]
[--input_size INPUT_SIZE] [--ratio RATIO]
[--batch_size BATCH_SIZE] [--shuffle]
[--flip] [--region]
[--load_thread LOAD_THREAD] [--crop CROP]
[--gray] [--gpu_ids GPU_IDS]
[--box_ratio BOX_RATIO]
[--box_scale BOX_SCALE]
[--input_channel INPUT_CHANNEL] [--mean MEAN]
[--std STD] [--padding]
[--checkpoint_name CHECKPOINT_NAME]
[--pretrain]
[--validate_ratio VALIDATE_RATIO]
[--sum_epoch SUM_EPOCH]
[--save_epoch_freq SAVE_EPOCH_FREQ]
[--save_batch_iter_freq SAVE_BATCH_ITER_FREQ]
[--lr LR] [--gamma GAMMA]
[--lr_mult_w LR_MULT_W]
[--lr_mult_b LR_MULT_B]
[--lr_policy LR_POLICY]
[--lr_decay_in_epoch LR_DECAY_IN_EPOCH]
[--momentum MOMENTUM]
[--weight_decay WEIGHT_DECAY]
[--loss_weight LOSS_WEIGHT] [--top_k TOP_K]
[--score_thres SCORE_THRES]
[--label_file LABEL_FILE]
[--classify_dir CLASSIFY_DIR]
[--display_winsize DISPLAY_WINSIZE]
[--display_id DISPLAY_ID]
[--display_port DISPLAY_PORT]
[--image_ncols IMAGE_NCOLS] [--html]
[--update_html_freq UPDATE_HTML_FREQ]
[--display_train_freq DISPLAY_TRAIN_FREQ]
[--display_validate_freq DISPLAY_VALIDATE_FREQ]
[--display_data_freq DISPLAY_DATA_FREQ]
[--display_image_ratio DISPLAY_IMAGE_RATIO]
multi_label_classifier.py: error: argument --dir is required
Process finished with exit code 2

i debuged,i found the code in multi_label_classifier.py
opt = op.parse()

i debuged step by step ,i found this line can't return a value''opt'" ,very strange.

the batch normalization layer of resnet output NaN

I tried the resnet50 layer as the classification model, and I changed the input size of the image, but the loss of validation step was nan. I found the problem was the batch normalization layer, the output of it was NaN. Do you have any idea about it?

Problem when I change the model to VGG, Resnet or Alexnet

When I change the model to the anythings other that Light.., it gives me an error in line 30 of model.py (missed match the input)
size mismatch, m1: [1 x 8192], m2: [25088 x 4096] at /pytorch/aten/src/TH/generic/THTensorMath.c:2033

FileNotFoundError

Unable to run the code, please help.
FileNotFoundError: [Errno 2] No such file or directory: '/data/image_server/images/CelebA/data/Align/003450.jpg'

https://drive.google.com/drive/folders/0B7EVK8r0v71pTUZsaXdaSnZBZzg is this the right dataset?

screenshot

About loss function

Hi @pangwong,
Thank you for sharing your project with us.
I saw that the loss function you're using for multilabel classification is CrossEntropyLoss function. I thought this function was only used for one-label classification. Could you explain how you used the CrossEntropyLoss function for multilabel classification? I'm trying to run my model for the NUSWIDE dataset using the MultiLabelSoftMarginLoss function, but the accuracy is getting pretty low.

CelebA dataset question

FileNotFoundError: [Errno 2] No such file or directory: '/data/image_server/images/CelebA/data/Align/026781.jpg'

Do you have the relative dataset? Could you please upload it?
And if I download from the official website, the dataset could works?

questions about code

your multi_label_classifier was constructed by several concurrent FC layers .Another idea is that the label is encoded by one-hot encoder and add a sigmoid function after the 3rd FC layer.At last ,only a CrossEntropyLoss is used without any concurrent FC layers.Do you think the idea works or have you try it?Thanks very much!