SparK for semantic segmentation about spark

[keyu-tian]

@simonebonato thank you.

I'm not an expert in semantic segmentation, but from what I know of some self-supervised papers, some will add a non-pre-trained UperNet head after a pre-trained backbone and then fine-tune them for semantic segmentation. Perhaps you could try to fine-tune our pre-trained ConvNeXt with https://github.com/facebookresearch/ConvNeXt/tree/main/semantic_segmentation.

For your idea, I believe it makes sense to pre-train a U-Net encoder. I also think it might be better if the whole UNet (including the decoder) is pre-trained together by SparK, but it'll require more efforts in implementation.

[simonebonato]

Ok then I can either try to take one of the pretrained ConvNeXt you provided, add a UperNet head and then finetune the entire thing, or I can try to use SparK directly on a a whole U-Net (encoder+decoder)?

I can try to do both and I will let you know in case I get something good :)

[keyu-tian]

Yeah they're two possible ways. Good luck with your experiments!