Is your model CNN or BCNN? about deep-bayesian-active-learning

[damienlancry]

I think it s a BCNN because of the dropout layers

[ShellingFord221]

But if so, then all the neural networks with just dropout layers can be called Bayesian neural networks? I don't think so!

[ShellingFord221]

As @damienlancry say, the experiment in section 5.2 is a comparison of CNN with dropout layers and CNN without dropout layers. But dropout layer is very common in recent models, who would use a CNN without dropout layers? It doesn't make sense!

[damienlancry]

Dropout is commonly used during training as a regularization technique to avoid overfitting since 2012. however it is not common to use dropout at test time. dropout layers are usually deactivated at test time thanks to model.eval() in pytorch for example. In a bayesian setting, dropout is never deactivated, we run several passes through the stochastic network to obtain a distribution of outputs and compute uncertainties.

[ShellingFord221]

Thanks! So in a word, a BCNN is a CNN using dropout layers at test time, but the same as common CNN when training?

[damienlancry]

I think so yeah

[ShellingFord221]

I have another question... In MC_Dropout_Keras/Dropout_Bald_Q10_N1000_Paper.py for example, the Bayesian part I think is shown in line 228:
dropout_score = model.predict_stochastic(X_Pool_Dropout,batch_size=batch_size, verbose=1)
, which measures the uncertainties. But I didn't find the implementation of the function predict_stochastic...

[damienlancry]

It is a method they implemented themselves, it is not implemented in the common keras available with pip, that is why they included their own version of keras in this repository
To use it, you have to add this keras to your python path. Personnally I simply added that to the beginning of every script:
import sys
sys,path.append("/home/damien/Documents/Deep-Bayesian-Active-Learning/keras")

[ShellingFord221]

@damienlancry Thank you so much for your kindly help! I still wonder that if there is only 1 new function, is it necessary to rewrite the whole keras? I don't know much about keras, sorry :(

[ShellingFord221]

Besides, is there any Pytorch version of BCNN? I think it will be a little bit easy in Pytorch to implement MC dropout when testing.