[enhancement] RAM management about g-diffuser-bot

[Ma5onic]

Since the queue executes commands sequentially, wouldn't it be more efficient to load models on-demand? That way people with a GPU with <8GB RAM in their GPU can load the bot.

[parlance-zz]

The beta2 branch already reduces this down to just 2 pipelines as the inpainting pipeline is a superset of the functionality built into img2img. Technically, it is also a superset of the functionality of the txt2img pipeline and I intend to reduce the number of pipes down to just 1, at least until diffusers can get their act together with memory usage.

I am leaving this issue open to track the progress. Thank you.

[parlance-zz]

I should also mention:

The model load time is very significant and can easily quadruple the generation time if you need to reload the model for each command. The workaround in beta2 is a) reducing the number of required pipes, and b) you can use the CLI to do single generation commands one at a time, and this will only use whatever model you need to actually complete your command, very similar to how the old CompVis code worked.

Lastly, in beta2 you have the option of using a pre-specified sub-selection of pipes in the config file, allowing you to save memory if you choose to forego the functionality in unneeded pipes. This will be important when more models / pipes are added to diffusers.

[parlance-zz]

This is now complete with the integration of haffried's grpc server in the dev branch.