How to train diffusion model on paired i2i dataset like style transfer? about joligen

[beniz]

If you have a paired dataset and want to use the palette model directly, there's and imminent PR by @royale coming up for improved super-resolution using a modified version of the aligned dataloader and that you could use (minus the upsampling step).
If you'd like to try sooner and hack the code a bit, there's a way to do it and we can point you to the modifications.

Side notes:

• the built-in GAN with semantics are powerful for style transfer, especially if you have annotations (e.g. object locations, masks) in your paired i2i dataset
• you can already train an 'inverse' inpaiting model (again if you have some object location labels): the objects are conserved while the other pixels are generated via diffusion. This may not be your exact use-case but worth mentioning IMO.

[beniz]

Pix2pix training with diffusion is now available, see https://www.joligen.com/doc/training.html#ddpm-training-for-pix2pix-task for an example.