Question about structure learning about proda

[theo2021]

Hi, the author can answer the question better, but this technique is commonly used in contrastive learning 1, 2. In contrastive learning, you train based on the idea that two different inputs that were produced by the same sample should lead to the exact output. It is a self-supervised technique that aims to create representations that are close to each other (for augmented samples).

Indeed sometimes the weak and strong are closer together compared to the strong and original. But in most of the cases, they are further apart, since the transformations are random, it is rare to have similar ones. Moreover, having weak transform ads further randomness that can benefit learning in the long run. In the author's case, the network learns a shared feature representation across augmentations while in the one you describe the network learns how to clean the transformations, so as to be as close as possible to an original image.

[super233]

Thanks for your answer, I'm a green hand in UDA. 🏃‍♂️