Keep xformers optimization in training. about sd_dreambooth_extension

[d8ahazard]

Yes, I believe if you pass the xformers command-line options, then it should install and be used with this.

I don't know if it will negatively impact training results - but it should definitely save VRAM. My motto is "try and find out!". :D

[sgsdxzy]

Yes, I believe if you pass the xformers command-line options, then it should install and be used with this.

I don't know if it will negatively impact training results - but it should definitely save VRAM. My motto is "try and find out!". :D

You write an unconditional sd_hijack.undo_optimization(), so I have to comment it out. Maybe adding an option to not undo optimizations?

[sgsdxzy]

It seems not to be the case, the optimization in ldm does not affect those used by diffuser, finguring out another way.

[d8ahazard]

It seems not to be the case, the optimization in ldm does not affect those used by diffuser, finguring out another way.

I just double-checked, and both shivam's repo and the default huggingface repo have the same code regarding xformers optimizations...meaning Shivam hasn't done anything special in this regard.

Just in case, I'll remove the sd_hijack undo_optimization() call, because it really shouldn't do anything here either way.

[d8ahazard]