How to decide the bandwidth? about s2cnn

[tscohen]

Ultimately it is a hyperparameter, similar to the spatial resolution in a planar CNN. In a planar CNN this is purely determined by by the stride of the convolutions and the pooling, but in spherical CNNs you have more flexibility: you can in principle choose the resolution freely in each layer.

There are currently no good "best practices" for spherical CNN architecture design, and this includes the bandwidth/resolution, but there are a couple of considerations that would factor into the decision:

• Higher resolution means you can represent more small details
• Higher resolution means higher computational cost
• If you reduce the resolution too quickly, you would ignore units in the input, just like when you use 2D convolution with a stride that is larger than the filter size.
• If the task is classification, you would typically start with a high resolution and gradually decrease it. The final layer can have very low resolution, and each unit has a receptive field that covers the whole input.
• If the task is e.g. segmentation, you could try a U-net like architecture.

We choose bandwidth=30 because it's not too large, but still allows us to represent MNIST digits without losing too much detail. MNIST images are 28x28, but we project them only on the top of the sphere, so using a spherical grid with 2*b=60 samples per dimension, we can represent it fairly accurately.

[Jiankai-Sun]

Thank you for your quick reply!

As the paper describes, We created two instances of this dataset: one in which each digit is projected on the northern hemisphere and one in which each projected digit is additionally randomly rotated. Why do you just project on the northern hemisphere instead of the entire sphere?

Thank you!

[Jiankai-Sun]

Also, for VGG Net, there are some nn.MaxPool2d(kernel_size=2, stride=2) layers. Is there any implementations of MaxPool2d() operation or MaxPool3d() operation for Spherical CNN? Probably so3_integrate() is one possible solution as this issue mentioned. However can we feed the output of so3_integrate as the input of the next SO3Convolution()? I am afraid the shape is not appropriate. Or can we directly use torch.nn.MaxPool2d / torch.nn.MaxPool3d same as standard 2D CNN?

Or probably we don't have to think about pooling at all since there is no kernel_size or stride concepts for s2cnn?

Thank you for your suggestion.

[tscohen]

We projected onto the northern hemisphere because that way the digit doesn't get stretched too much. It's just a toy experiment so we didn't think about this too much. Projecting it on the whole sphere would most likely work as well.

Max pooling is a bit tricky. You could just do nn.MaxPool2d or 3d on the array that stores the feature map, but due to the inhomogeneous sampling grid, this would not be equivariant. It would probably still be approximately equivariant, and may work in practice.

so3_integrate() does a global average pooling. If you want to do a local average pooling, you could use a convolution with a fixed Gaussian blur filter, and sample the result on a low-resolution (low-bandwidth) grid.

[Jiankai-Sun]

Great thanks to your quick reply! Your work is so fascinating!