Coder Social home page Coder Social logo

cvjena / part_constellation_models Goto Github PK

View Code? Open in Web Editor NEW
47.0 11.0 22.0 1.21 MB

Code for the ICCV 2015 paper "Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks."

MATLAB 100.00%
caffe fine-grained-classification part-models constellation-models neural-activation-constellations

part_constellation_models's Introduction

Part Constellation Models

This is the code used in our paper "Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks" by Marcel Simon and Erik Rodner published at ICCV 2015. If you would like to refer to this work, please cite the corresponding paper

@inproceedings{Simon15:NAC,
author = {Marcel Simon and Erik Rodner},
booktitle = {International Conference on Computer Vision (ICCV)},
title = {Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks},
year = {2015},
}

The following steps will guide you through the usage of the code.

1. Setup

  1. Open Matlab and go to the folder containing this package
  2. Run setup.m to download all libraries
  3. Go to lib/caffe_pp and make it, you will need to create a Makefile.config. If you have an existing caffe, use that Makefile.config from there BUT DO NOT USE ANY EXISTING CAFFE as caffe_pp is a modified version.
  4. Execute make mat in lib/caffe_pp
  5. Go to lib/liblinear-2.1 and make it
  6. Go to lib/liblinear-2.1/matlab and make it

2. Running the code

The script.m in the root folder of the package is all you need. You want to override the paths to the data set by passing them as name-value-pairs, for example start('basedir','/path/to/dataset/'). For more options, open it to see all options. Just pass additional parameters by adding name-value-pairs: start('basedir','/path/to/dataset/','cnn_dir','./cnn_finetuning/vgg19/','crop_size',224);.

The dataset files should contain a list of absolute image paths, a list of corresponding labels starting from 1, and a list of the corresponding assignment to train and test, where 1 indicates training and 0 test.

imagelist.txt

/path/to/image1.jpg
/path/to/image2.jpg
/path/to/image3.jpg
/path/to/image4.jpg
/path/to/image5.jpg
...

labels.txt

1
1
1
2
2
2
...

tr_ID.txt

0
1
1
0
1
1
...

3. Testing the models from the paper

The models of the paper are available at https://drive.google.com/file/d/0B6VgjAr4t_oTQXN2Y3VYaEMwVDA/view?usp=sharing. Download and unzip them to the root folder of the code. You can run them by executing, for example, start('cache_dir','./cache_iccv_cub200','cnn_dir','./cnn_finetuning/vgg19/','crop_size',224,'basedir','/home/simon/Datasets/CUB_200_2011/').

License

The Part Constellation Models Framework by Marcel Simon and Erik Rodner is licensed under the non-commercial license Creative Commons Attribution 4.0 International License. For usage beyond the scope of this license, please contact Marcel Simon.

part_constellation_models's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

part_constellation_models's Issues

Error using caffe_prepare_image (line 25) Subscript indices must either be real positive integers or logicals.

Hi,
I am trying to run training process by the following command.
start('basedir','/path/to/CUB_200_2011_dataset/','cnn_dir','./cnn_finetuning/vgg19/','imagedir','/path/to/CUB_200_2011_dataset/images');.

However, I got the error below:


Starting parallel pool (parpool) using the 'local' profile ... "/auto/matlab-2015b/toolbox/distcomp/bin/util/psname.sh" GLNXA64 1074: Aborted
Image 11788: 200.Common_Yellowthroat/Common_Yellowthroat_0055_190967.jpg
Error using caffe_prepare_image (line 25)
Subscript indices must either be real positive integers or logicals.

Error in caffe_gradients (line 39)
    im = caffe_prepare_image(im,mean,width);

Error in parts_locs_from_grads (line 25)
    parfor (i = 1:size(imagelist,1), opts.parfor_arg)

Error in start (line 162)
    part_locs = parts_locs_from_grads(opts);

I generated the txt files by following the tutorial. Have you ever run into this problem?

Error using vertcat: Dimensions of matrices being concatenated are not consistent.

Hi,
I'm trying to run the code with your pre-trained models. I use the following command in MATLAB terminal:

start('cache_dir', 'Simon_ICCV15_models/cache_iccv_cub200', ...
      'cnn_dir', 'Simon_ICCV15_models/cnn_finetuning/vgg19/',...
      'crop_size', 224);

Then I got the output and an error like:

opts = 

  struct with fields:

                  caffe_path: './lib/caffe_pp'
              liblinear_path: './lib/liblinear-2.1/matlab/'
              selsearch_path: './lib/SelectiveSearchCodeIJCV/'
                    imagedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images'
                     basedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011'
                   cache_dir: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cache/'
              store_features: 0
                     cnn_dir: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cnn_finetuning/caffe_reference/'
                  batch_size: 11
                   crop_size: 227
                  finetuning: 1
            finetuning_iters: 10000
              finetuning_gpu: 0
               estimate_bbox: 1
              calculate_mean: 1
                  part_layer: 'pool5'
    part_layer_channel_count: 256
           no_selected_parts: 10
            no_visible_parts: 5
                  view_count: 10
                  iterations: 5
                  class_wise: 1
              feature_global: 'relu7'
                 use_flipped: 1
            use_bounding_box: 1
              pyramid_levels: 0
              rand_tr_images: -1
                rand_tr_part: 0
                 repetitions: 1
                   use_parts: 1
                 part_scales: [0.4400 0.2400]
                feature_part: 'relu7'
                  svm_params: '-q'
              parfor_workers: 2
                  use_parfor: 1
              verbose_output: 1
              imagelist_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images.txt'
                  tr_ID_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/train_test_split.txt'


opts = 

  struct with fields:

                  caffe_path: './lib/caffe_pp'
              liblinear_path: './lib/liblinear-2.1/matlab/'
              selsearch_path: './lib/SelectiveSearchCodeIJCV/'
                    imagedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images'
                     basedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011'
                   cache_dir: 'Simon_ICCV15_models/cache_iccv_cub200'
              store_features: 0
                     cnn_dir: 'Simon_ICCV15_models/cnn_finetuning/vgg19/'
                  batch_size: 11
                   crop_size: 224
                  finetuning: 1
            finetuning_iters: 10000
              finetuning_gpu: 0
               estimate_bbox: 1
              calculate_mean: 1
                  part_layer: 'pool5'
    part_layer_channel_count: 256
           no_selected_parts: 10
            no_visible_parts: 5
                  view_count: 10
                  iterations: 5
                  class_wise: 1
              feature_global: 'relu7'
                 use_flipped: 1
            use_bounding_box: 1
              pyramid_levels: 0
              rand_tr_images: -1
                rand_tr_part: 0
                 repetitions: 1
                   use_parts: 1
                 part_scales: [0.4400 0.2400]
                feature_part: 'relu7'
                  svm_params: '-q'
              parfor_workers: 2
                  use_parfor: 1
              verbose_output: 1
              imagelist_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images.txt'
                  tr_ID_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/train_test_split.txt'
                 labels_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/image_class_labels.txt'
                      deploy: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cnn_finetuning/caffe_reference//deploy.prototxt'
                       model: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cnn_finetuning/caffe_reference//model'
               mean_mat_file: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cnn_finetuning/caffe_reference//mean.mat'
             mean_proto_file: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cnn_finetuning/caffe_reference//mean.binaryproto'
            caffe_executable: './lib/caffe_pp/build/tools/caffe'
              finetuning_dir: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/cache//cnn_finetuning/caffe_reference'

Warning: Directory already exists. 
> In start (line 137)
  In run_demo (line 1) 

opts = 

  struct with fields:

                  caffe_path: './lib/caffe_pp'
              liblinear_path: './lib/liblinear-2.1/matlab/'
              selsearch_path: './lib/SelectiveSearchCodeIJCV/'
                    imagedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images'
                     basedir: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011'
                   cache_dir: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200'
              store_features: 0
                     cnn_dir: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cnn_finetuning/vgg19/'
                  batch_size: 11
                   crop_size: 224
                  finetuning: 1
            finetuning_iters: 10000
              finetuning_gpu: 0
               estimate_bbox: 1
              calculate_mean: 1
                  part_layer: 'pool5'
    part_layer_channel_count: 256
           no_selected_parts: 10
            no_visible_parts: 5
                  view_count: 10
                  iterations: 5
                  class_wise: 1
              feature_global: 'relu7'
                 use_flipped: 1
            use_bounding_box: 1
              pyramid_levels: 0
              rand_tr_images: -1
                rand_tr_part: 0
                 repetitions: 1
                   use_parts: 1
                 part_scales: [0.4400 0.2400]
                feature_part: 'relu7'
                  svm_params: '-q'
              parfor_workers: 2
                  use_parfor: 1
              verbose_output: 1
              imagelist_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/images.txt'
                  tr_ID_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/train_test_split.txt'
                 labels_file: '/home/duang/Documents/GraduateProject/Datasets/CUB-200-2011/CUB_200_2011/CUB_200_2011/image_class_labels.txt'
                      deploy: 'Simon_ICCV15_models/cnn_finetuning/vgg19//deploy.prototxt'
                       model: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cnn_finetuning/vgg19//model'
               mean_mat_file: 'Simon_ICCV15_models/cnn_finetuning/vgg19//mean.mat'
             mean_proto_file: 'Simon_ICCV15_models/cnn_finetuning/vgg19//mean.binaryproto'
            caffe_executable: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/lib/caffe_pp/build/tools/caffe'
              finetuning_dir: 'Simon_ICCV15_models/cache_iccv_cub200/cnn_finetuning/vgg19'
               part_loc_file: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/part_locs.mat'
               est_bbox_file: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/est_bbox.txt'
     caffe_window_file_train: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/windows_train.txt'
       caffe_window_file_val: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/windows_val.txt'
            caffe_part_model: '/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/part_model.mat'

Loading part locs from /home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/Simon_ICCV15_models/cache_iccv_cub200/part_locs.mat
Loading part model from cache...
Loading estimated bounding boxes and region proposals from disk...
Using pretrained model...
Starting classification...
Warning: File:
/home/duang/Documents/GraduateProject/Codes/Constellations/part_constellation_models/part_based_classification/part_box_classification_multiscale.m
Line: 143 Column: 30
The temporary variable cur_box will be cleared at the beginning of each
iteration of the parfor loop.
Any value assigned to it before the loop will be lost.  If cur_box is used
before it is assigned in the parfor loop, a runtime error will occur.
See Parallel for Loops in MATLAB, "Temporary Variables". 
> In start (line 212)
  In run_demo (line 1) 

Error using vertcat
Dimensions of matrices being concatenated are not consistent.

Error in part_box_classification_multiscale (line 86)
        labels = [labels;labels(tr_ID)];

Error in start (line 212)
part_box_classification_multiscale( channel_ids(1:opts.no_selected_parts),
part_locs, opts );

Error in run_demo (line 1)
start('cache_dir', 'Simon_ICCV15_models/cache_iccv_cub200', ...

I ran the code again and printed the dimensions of some key params:

  Name           Size            Bytes  Class      Attributes

  tr_ID      11788x2             23576  logical              

  Name            Size             Bytes  Class     Attributes

  labels      11788x2             188608  double              

  Name               Size             Bytes  Class     Attributes

  labels(tr_ID)      17782x1             142256  double              

I wonder if you used the original CUB-200-2011 dataset, or maybe you have modified the format of the txt files in the dataset.
I mean, for example, the first column of the original CUB-200-2011 dataset's txt files are the line numbers, the second column are the values. Maybe you deleted the first column and just left the second column.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.