visualize VGG16 activations about keract

[philipperemy]

@waspinator looks great! I'm going to work on something. Display_activations is not very robust.

[MJamali89]

Hi,
if I want to show the output in gray, is it possible or not? what should I do for this?
Thank you.

[philipperemy]

import matplotlib.pyplot as plt

plt.imshow(matrix, cmap='gray')

@nadianaji

https://stackoverflow.com/questions/3823752/display-image-as-grayscale-using-matplotlib

[MJamali89]

I used your keract.display_activation function, is it possible to show my activations in gray using this function?

[philipperemy]

Nope you'll have to edit it yourself. It's in my plan to change keract.display_activation

[MJamali89]

Thank you. it is a very good function and before that, I had several problems for visualizing the output of intermediate layers. you said before, there may be some issue with display_activation. Does it mean it maybe show the wrong thing?

[MJamali89]

sorry, what is the name used in activations.get(block1_conv1/Relu:0) function? are they the name of layers? because when I use the name of the layer it produces TypeError: 'NoneType' object is not subscriptable

[MJamali89]

I put here my code and I have some models in my code when I want to use keract.get_activation it produces this error. what is the problem and how can I solve it?

AttributeError: Layer encoder has multiple inbound nodes, hence the notion of "layer output" is ill-defined. Use get_output_at(node_index) instead.

from keras.layers import Input, Concatenate, GaussianNoise,Dropout
from keras.layers import Conv2D
from keras.models import Model
from keras.datasets import mnist
from keras.callbacks import TensorBoard
from keras import backend as K
from keras import layers
import matplotlib.pyplot as plt
import tensorflow as tf
import keras as Kr
import keras
import numpy as np
import pylab as pl
import matplotlib.cm as cm
import keract
from keras import optimizers
from keras import regularizers
from keras.callbacks import EarlyStopping

from tensorflow.python.keras.layers import Lambda;

#-----------------building w train---------------------------------------------
w_main = np.random.randint(2,size=(1,14,14,1))
w_main=w_main.astype(np.float32)
w_expand=np.zeros((1,28,28,1),dtype='float32')
w_expand[:,0:14,0:14]=w_main
w_expand.reshape(1,28,28,1)
w_expand=np.repeat(w_expand,49999,0)

#-----------------building w validation---------------------------------------------
w_valid = np.random.randint(2,size=(1,14,14,1))
w_valid=w_valid.astype(np.float32)
wv_expand=np.zeros((1,28,28,1),dtype='float32')
wv_expand[:,0:14,0:14]=w_valid
wv_expand.reshape(1,28,28,1)
wv_expand=np.repeat(wv_expand,9999,0)

#-----------------building w test---------------------------------------------
w_test = np.random.randint(2,size=(1,14,14,1))
w_test=w_test.astype(np.float32)
wt_expand=np.zeros((1,28,28,1),dtype='float32')
wt_expand[:,0:14,0:14]=w_test
wt_expand.reshape(1,28,28,1)
#wt_expand=np.repeat(wt_expand,10000,0)

#-----------------------encoder------------------------------------------------
#------------------------------------------------------------------------------
wtm=Input((28,28,1),name='watermark')
image = Input((28, 28, 1),name='image')
conv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name='convl1e')(image)
conv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name='convl2e')(conv1)
conv3 = Conv2D(8, (3, 3), activation='relu', padding='same', name='convl3e')(conv2)
#conv3 = Conv2D(8, (3, 3), activation='relu', padding='same', name='convl3e', kernel_initializer='Orthogonal',bias_initializer='glorot_uniform')(conv2)
DrO1=Dropout(0.25,name='Dro1')(conv3)
encoded =  Conv2D(1, (3, 3), activation='relu', padding='same',name='reconstructed_I')(DrO1)


#-----------------------adding w---------------------------------------
#add_const = Kr.layers.Lambda(lambda x: x + Kr.backend.constant(w_expand))
encoded_merged=keras.layers.Add()([encoded,wtm])

#add_const = Kr.layers.Lambda(lambda x: x[0] + x[1],name='lambda')
#encoded_merged = add_const([encoded,wtm])
encoder=Model(inputs=[image,wtm], outputs= encoded_merged ,name='encoder')
encoder.summary()

#-----------------------decoder------------------------------------------------
#------------------------------------------------------------------------------
deconv_input=Input((28,28,1),name='inputTodeconv')
#encoded_merged = Input((28, 28, 2))
deconv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name='convl1d',kernel_regularizer=regularizers.l2(0.001), kernel_initializer='Orthogonal')(deconv_input)
deconv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name='convl2d')(deconv1)
deconv3 = Conv2D(8, (3, 3), activation='relu',padding='same', name='convl3d')(deconv2)
DrO2=Dropout(0.25,name='DrO2')(deconv3)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same', name='decoder_output')(DrO2) 

decoder=Model(inputs=deconv_input, outputs=decoded, name='decoder')
#decoder.summary()
encoded_merged = encoder([image,wtm])
decoded = decoder(encoded_merged)

model=Model(inputs=[image,wtm],outputs=decoded,name='firstpartmodel')
#----------------------w extraction------------------------------------
convw1 = Conv2D(16, (3,3), activation='relu', padding='same', name='conl1w',kernel_regularizer=regularizers.l2(0.001), kernel_initializer='Orthogonal')(decoded)
convw2 = Conv2D(32, (3, 3), activation='relu', padding='same', name='convl2w')(convw1)
convw3 = Conv2D(8, (3, 3), activation='relu', padding='same', name='conl3w')(convw2)
DrO3=Dropout(0.25, name='DrO3')(convw3)
pred_w = Conv2D(1, (1, 1), activation='sigmoid', padding='same', name='reconstructed_W')(DrO3)  
# reconsider activation (is W positive?)
# should be filter=1 to match W
w_extraction=Model(inputs=[image,wtm],outputs=[decoded,pred_w],name='mainModel')

#----------------------training the model--------------------------------------
#------------------------------------------------------------------------------
#----------------------Data preparesion----------------------------------------

(x_train, _), (x_test, _) = mnist.load_data()
x_validation=x_train[1:10000,:,:]
x_train=x_train[10001:60000,:,:]
#
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_validation = x_validation.astype('float32') / 255.
x_train = np.reshape(x_train, (len(x_train), 28, 28, 1))  # adapt this if using `channels_first` image data format
x_test = np.reshape(x_test, (len(x_test), 28, 28, 1))  # adapt this if using `channels_first` image data format
x_validation = np.reshape(x_validation, (len(x_validation), 28, 28, 1))

#---------------------compile and train the model------------------------------
# is accuracy sensible metric for this model?
adadelta=optimizers.Adadelta(lr=1.0,decay=1/1000)
w_extraction.compile(optimizer=adadelta, loss={'decoder':'mse','reconstructed_W':'mse'}, metrics=['mae'])
w_extraction.fit([x_train,w_expand], [x_train,w_expand],
          epochs=5,
          batch_size=32, 
          validation_data=([x_validation,wv_expand], [x_validation,wv_expand]),
          callbacks=[TensorBoard(log_dir='E:/tmp/AutewithW200', histogram_freq=0, write_graph=False),EarlyStopping(monitor='val_loss', patience=10,min_delta=0)])
model.summary()
#model.fit([images, w], [images, w], batch_size=64, epochs=5)
#--------------------visuallize the output layers------------------------------
activations = keract.get_activations(w_extraction, [x_test[8000:8001],wt_expand])
keract.display_activations(activations)

[philipperemy]

@nadianaji it's correct but not very robust. I'll have a look!

[philipperemy]

cf4a055 @waspinator thanks going to close this issue.
@nadianaji feel free to open a new one.