Bug in forward? about glom-pytorch

[A7ocin]

Yes I have, it works. I'm currently trying to figure out how to improve the model

[lucidrains]

@A7ocin Hi! There wasn't strictly a bug because the variables never got used, but I removed it to avoid any confusion!

[A7ocin]

Oh, I see, my bad! That's better anyway :)

[A7ocin]

Hi, I'm reopening this again because there is something that is not clear to me.
In the README, you give some examples, in particular, when using 6 levels:

top_level = all_levels[7, :, :, -1] # get the top-level embeddings after iteration 6

which sounds good. However, in the network, you initialise the number of iterations as double the number of levels, for the top-down and bottom-up information to propagate. So my question is: why do we get the top-level embeddings at iteration 7 and not, let's say, 12?
Thank you for your patience

[lucidrains]

@A7ocin Hi Nicola again! I may not be thinking clearly, but my reasoning was that that is how long it would take for the data to propagate all the way up the layers and back down to the start. So say you were doing some segmentation task akin to https://arxiv.org/abs/2103.13413 , then you could attach losses to the bottom layer as well. If you are only doing losses on the top layer, 6 iterations is enough!

[lucidrains]

@A7ocin I see some connections between GLOM and the feature pyramids commonly used in vision nets

[pfabreu]

@lucidrains sorry to hijack this issue, but have you (or anyone else) tried training this as a denoising autoencoder on MNIST or CIFAR?

[pfabreu]

Could you please share your code/training loop? I think an interesting idea would be to make the autoencoders VAEs (the embeddings would be distribution parameters and sampled from at inference time), for a purely generative model.

[MrChill]

Yes I have, it works. I'm currently trying to figure out how to improve the model

@A7ocin @lucidrains What did you change? what patch size did you use?