About sparse convolution about spark

[keyu-tian]

1. We used masked operators or index-select operators to simulate truly sparse conv or sparse batchnorm, etc, since we found they were faster in practice.
2. This is an interesting perspective, and the problem could exist in all masked autoencoding algorithms. I think data augmentation (we use random flipping and cropping) can mitigate this.
3. they are removed and we only use the pretrained encoder.

[Itsanewday]

Thanks for your fast and kindly reply! And, I have another question that about finetuning for semantic segmentation, should i frozen the encoder?

[keyu-tian]

you can refer to what we did in object detection and instance segmentation:

SparK/downstream_d2/lr_decay.py

Line 8 in 00883b8

def lr_factor_func(para_name: str, is_resnet50, dec: float, debug=False) -> float:

.

In this function we split a resnet encoder into N chunks and scale each parameter's learning rate based on which chunks it belongs. The i-th chunk has a scaling ratio of dec ** (N-i). So deeper chunks will have larger learning rates than shallow ones. The end chunk will have 100% the learning rate. So actually we won't frozen any parameter.

[Itsanewday]

Thank you very much!