Relu activation in PrimaryCap? about capsnet-tensorflow

[naturomics]

In fact, I have the same doubts. I've read the paper again and again, but I can't find out any words in the paper whether PrimaryCaps has a ReLU activation function or not. So I just let it go. Maybe I should write some comments in the code to note that. I'll try it out which is better, and use it as the default.

Thank you for your comment

[naturomics]

@oargueta3 I've done a experiment for this doubts. The results show that using ReLU has a higher test accuracy(batch size 48, 50 epoch,~99.4) than no ReLU(batch size 48, 30 epoch, the best result is 99.1 at step 8600, but most time it maintains at 99.0)

[stoneyang]

@naturomics , thanks for your work, and it is great and really quick!

It seems that there's no ReLU between capsule and squashing. As mentioned in the third paragraph in Sec. 4 of the original paper, both v1 and v2:

One can see PrimaryCapsules as a Convolutional layer with Eq. 1 as its block non-linearity.

So, I think we may confirm that no ReLU is placed there.

Any suggestions?