cyril9227 / keras_attentivenormalization Goto Github PK

It appears this implementation only supports 4D inputs - though Conv1D takes and outputs 3D tensors; I found that making below changes enables AN to work for 3D tensors without errors:

GlobalAveragePooling2D() --> GlobalAveragePooling1D()
gamma_readjust[:, None, None, :] --> gamma_readjust[:, None, :]
beta_readjust[:, None, None, :] --> beta_readjust[:, None, :] 

I'm unsure, however, whether this is a correct implementation consistent with the paper - can you confirm? Thanks

Hi There,

I have tried to run the test script several times but not running.

TypeError Traceback (most recent call last)
in
1 inp = Input((64, 64, 3))
2 x = Dense(20, activation="relu")(inp)
----> 3 x = AttentiveNormalization(n_mixture=5, momentum=0.99, epsilon=0.001, axis=-1)(x)

TypeError: list indices must be integers or slices, not ListWrapper

My configurations:
python 3.6
TensorFlow-GPU 2.0.0
Keras 2.2.4

TypeError Traceback (most recent call last)
in
12 x = Dense(20, activation="relu")(inp)
13 print(x.shape)
---> 14 x = AttentiveNormalization(n_mixture=5, momentum=0.99, epsilon=0.001, axis=-1)(x)
15 x = GlobalAveragePooling2D()(x)
16 pred = Dense(1, activation='sigmoid')(x)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py in wrapper(*args, **kwargs)
690 except Exception as e: # pylint:disable=broad-except
691 if hasattr(e, 'ag_error_metadata'):
--> 692 raise e.ag_error_metadata.to_exception(e)
693 else:
694 raise

TypeError: Exception encountered when calling layer "attentive_normalization" (type AttentiveNormalization).

in user code:

File "/content/drive/MyDrive/fall22sem/AN/AttentiveNormalization.py", line 57, in call  *
    out_BN = super(AttentiveNormalization, self).call(input) # rescale input, N x H x W x C
File "/usr/local/lib/python3.7/dist-packages/keras/layers/normalization/batch_normalization.py", line 784, in call  **
    reduction_axes = [i for i in range(ndims) if i not in self.axis]
File "/usr/local/lib/python3.7/dist-packages/keras/layers/normalization/batch_normalization.py", line 784, in <listcomp>
    reduction_axes = [i for i in range(ndims) if i not in self.axis]

TypeError: argument of type 'int' is not iterable

Call arguments received:
• input=tf.Tensor(shape=(None, 64, 64, 20), dtype=float32)

First, thanks for sharing this work.But I have some confusion, as what I show in this pic. Do you mean that for each batch(like N_0), for each channel in this batch.the weight(the dimension is K) is the same?Because I also notice that the shape of the output of the sigmoid is the (N*K).
Thanks for your answer.