iamhankai / attribute-aware-attention Goto Github PK

[ACM MM 2018] Attribute-Aware Attention Model for Fine-grained Representation Learning

Home Page: https://arxiv.org/abs/1901.00392

Python 97.92% Shell 2.08%

cub-dataset fine-grained-classification person-reidentification fine-grained-visual-categorization attention-mechanism representation-learning convolutional-neural-networks

attribute-aware-attention's People

Contributors

Stargazers

Watchers

attribute-aware-attention's Issues

Corresponding Tensorflow version

What is the corresponding tensorflow version for Keras 1.2.1?

inconsistence about final feature between paper and code?

Dear @iamhankai ,
Final feature equals region feature plus attr feature in your paper, while its equals concat of the two in your code. Is there any experiment shows which is better?

how to train without attribute label

Hi,

from the table(1) in the paper, you mentioned that the network can be trained without attribute label.

in such manner, what is the form of the final loss function? Do we only need to care about the categorization loss?

Running Command:

python cub_demo.py ResNet50 /home/dell/Videos/Object_Tracking/attribute-aware-atte
ntion/dataset/CUB_200_2011

Requirements:

pip install keras==1.2.1
pip install theano

added additionally, for backend set as 'theano'

import os
os.environ['KERAS_BACKEND'] = 'theano'

Error 1: ImportError: cannot import name inplace_increment

Deleting the cache manually worked.
rm -rf ~/.theano

Error 2: AttributeError: 'module' object has no attribute 'ifelse'

Your version of theano is probably too new for that version of Keras. You should try downgrading theano to 0.9.x,
pip install theano==0.9.0

Error 3: ValueError: Negative dimension size caused by subtracting 3 from 2 for 'MaxPool' (op: 'MaxPool') with input shapes: [?,2,2240,64].

i followed @iamhankai sir suggestion . changed "image_data_format" to "channels_first".
sudo gedit $HOME/.keras/keras.json

{
    "floatx": "float32",
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "image_data_format": "channels_first"
}.

Error 4: ValueError: total size of new array must be unchanged

Traceback (most recent call last):
  File "cub_demo.py", line 102, in <module>
    share_fea_map = Reshape((final_dim, L), name='reshape_layer')(share_fea_map)        
  File "/home/dell/Videos/Object_Tracking/env_a3m/local/lib/python2.7/site-packages/keras/engine/topology.py", line 572, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/home/dell/Videos/Object_Tracking/env_a3m/local/lib/python2.7/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/home/dell/Videos/Object_Tracking/env_a3m/local/lib/python2.7/site-packages/keras/engine/topology.py", line 170, in create_node
    output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
  File "/home/dell/Videos/Object_Tracking/env_a3m/local/lib/python2.7/site-packages/keras/layers/core.py", line 373, in get_output_shape_for
    self.target_shape)
  File "/home/dell/Videos/Object_Tracking/env_a3m/local/lib/python2.7/site-packages/keras/layers/core.py", line 367, in _fix_unknown_dimension
    raise ValueError(msg)
ValueError: total size of new array must be unchanged

I saw @iamhankai suggestion for this issue. but not understand clearly. I have few doubts, please help me @iamhankai @Muneeb699 sir. it is my request.

The input image size is 448448, not 224224.

here mentioned input image size/shape, exactly what are the width and height?
where I need to change the code?
either dataset/CUB_200_2011/images, all images need to resize?

Please guide me to solve this issue @iamhankai, sir.

Thanks and Regards
Murugan Rajenthiran

ValueError: Input 0 is incompatible with layer batchnormalization_59: expected ndim=2, found ndim=3

How to solve this problem ?

After epoch 11/50 , Main acc only about 0.02

ValueError: Shape must be rank 2 but is rank 3 for 'MatMul_29' (op: 'MatMul') with input shapes: [?,196,512], [?,512].

Hi @iamhankai. When I ran your code I got the following error:

File "/home/t0c01zx/anaconda3/envs/keras/lib/python3.6/site-packages/keras/engine/topology.py", line 1409, in call
output = K.batch_dot(l1, l2, self.dot_axes)
File "/home/t0c01zx/anaconda3/envs/keras/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 884, in batch_dot
out = tf.matmul(x, y, adjoint_a=adj_x, adjoint_b=adj_y)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 2455, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5333, in mat_mul
name=name)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
op_def=op_def)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1823, in init
control_input_ops)
File "/home/t0c01zx/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
raise ValueError(str(e))
ValueError: Shape must be rank 2 but is rank 3 for 'MatMul_29' (op: 'MatMul') with input shapes: [?,196,512], [?,512].

I was using Keras 1.2.1 and Tensorflow 1.13.0. The model I choose is ResNet50.

Looking forward to your reply!

Could u pls give us a download link for your paper?

It's hard to get the paper from ACM DL.

Model Folder is empty

Why the model folder is empty and where is model reside? I can't able to find it

How does sys.argv get net and data_loader?

Dear sir : when i run the code, sys.argv[0] only returns the absolutely path of cub_demo.py, sys.argv[1] returns error (list out of range). How can i deal?

two questions

Thank you for your beautiful paper and code, but i have two question when i did your experiment

1.it's my first time run keras' code, when i run run.sh "ValueError: Negative dimension size caused by subtracting 3 from 2 for 'MaxPool' (op: 'MaxPool') with input shapes: [?,2,2240,64]." I don't know why. my log is as below:
model_raw = eval(net)(input_tensor=inputs, include_top=False, weights='imagenet')
File "/usr/local/lib/python2.7/dist-packages/keras/applications/resnet50.py", line 179, in ResNet50
x = MaxPooling2D((3, 3), strides=(2, 2))(x)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 572, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 635, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 166, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/usr/local/lib/python2.7/dist-packages/keras/layers/pooling.py", line 160, in call
dim_ordering=self.dim_ordering)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/pooling.py", line 210, in _pooling_function
pool_mode='max')
File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 2786, in pool2d
x = tf.nn.max_pool(x, pool_size, strides, padding=padding)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1958, in max_pool
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 2806, in _max_pool
data_format=data_format, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 3 from 2 for 'MaxPool' (op: 'MaxPool') with input shapes: [?,2,2240,64].

2.what's mean about parameter "nb_attributes",I want do experiment in subset of CUB, should i change it?

How you are deciding this value 11788? and what does it mean?

transform binary attribute to clustered attribute

nb_attr = len(start_idxs)-1
A_all = np.zeros((11788,nb_attr))
image_attribute_file = '../CUB_200_2011/attributes/image_attribute_labels.txt'

ValueError: Negative dimension size caused by subtracting 3 from 2 for 'MaxPool' (op: 'MaxPool') with input shapes: [?,2,224,64].

when i run run.sh.it happened.what should i do?

processed_attributes.txt for CompCars Dataset

Hi @iamhankai, I'm trying to reproduce the A3M results on the CompCars dataset. I'm wondering how to generate processed_attributes.txt (Link) because I don't see attributes.txt for this dataset online. If you have it available, could you please share the necessary files(ex: attributes, processed_attributes, class_attribute_label_continuos) for CompCars required to reproduce results.

Thanks for the code to this amazing paper!!

Seek for a PyTorch version

Focused on the repo recently. Is there a version implemented by PyTorch？

processed_attribute_file.txt is mixing

How to include in our Project?

Hello I have got intrigued on seeing this paper:
I'm trying to make a object tracker I would like to use your work along with detection using transformers or with Re3 in order to reduce errors due to occlusion correct me if I am wrong how can your work be injected into them I'm trying to understand but with your support it will be easier to use your work while DETR is transformer based object detection and Re3 is LSTM based generic Object tracking algorithm both of them depend on attention mechanism all I need to understand is A3M attention in them, I know I am asking for a lot but help me out.

Thanks,
Rajashekar

A little problem about merge function

Thanks for your sharing and paper, which showed us an excellent idea in pedestrian re-identification. However, when I run your sample code with your sample dataset, something wrong in the function keras.layers.merge()

And the error is TypeError: 'module' object is not callable

First, I thought my tensorflow or keras environment have some wrong or conda have problems. But after long time research and testing, I found the problem is not caused by environment or system. Then I try to find the introduction of function merge, but I can only find information about this LAYER rather than MERGE FUNCTION identity! ( https://keras.io/layers/merge/ ) Finally, I got the solution here:

keras-team/keras#3921 (comment) .

It seems this layer is not exist anymore.
Then I changed code from merge(.., mode = "", dot_axis=**) to dot (.., axis = **) (or average, concatenate) and now it works.

It seems you need to upgrade your code as soon as possible and solve this little but annoying problem.
Thanks~

ValueError: Negative dimension size caused by subtracting 2 from 1 for 'MaxPool_1' (op: 'MaxPool') with input shapes: [?,1,224,128].

I have changed image_data_format in keras.json , but still says this

ValueError: Dimension 0 in both shapes must be equal, but are 7 and 64. Shapes are [7,7,448,64] and [64,3,7,7]. for 'Assign' (op: 'Assign') with input shapes: [7,7,448,64], [64,3,7,7].