woctezuma / stable-diffusion-colab

Colab notebook for Stable Diffusion Hyper-SDXL.

Stable Diffusion Hyper-SDXL - Colab

The goal of this repository is to provide a Colab notebook to run the text-to-image Stable Diffusion Hyper-SDXL model.

Usage

• Run stable_diffusion.ipynb.

Results

_{Different results obtained with the text prompt: "a photo of an astronaut riding a horse on Mars" using Stable Diffusion XL.}

---

_{Different results obtained with the text prompt: "a photo of Pikachu fine dining with a view to the Eiffel Tower" using Stable Diffusion XL.}

References

[1] Rombach, Robin, et al. High-resolution image synthesis with latent diffusion models. CVPR 2022. (models & demo)

[2] Karras, Tero, et al. Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS 2022. (code)

[3] Luo, Simian, et al. Latent Consistency Models: Synthesizing HR Images with Few-Step Inference. arXiv 2023. (code)

[4] Luo, Simian, et al. LCM-LoRA: A Universal Stable-Diffusion Acceleration Module. arXiv 2023.

stable-diffusion-colab's Issues

Textual Inversion for Stable Diffusion XL notebook

To Whom It May Concern:
I have published a stand-alone textual inversion implementation for Stable Diffusion XL - NO additional frameworks, just a Jupyter notebook, works on your local 24GB GPU e.g. RTX 3090:
https://github.com/oss-roettger/XL-Textual-Inversion

No module named 'triton'" / CUDA Error

Thanks for making this.

I have two issue:

1. I get this error:

"WARNING:root:A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'"

But I was able to generate images without any issue.

2. After generating the image, there's no way to free up GPU memory and getting cuda memory error when trying to generate more high resolution images. someone suggested this:

from numba import cuda
import torch 
device = cuda.get_current_device()
device.reset()

and it did empty the gpu memory but somehow detach the cuda device from the notebook!

CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Switch model weights to stabilityai/stable-diffusion-2-1

Switch model weights to stabilityai/stable-diffusion-2-1 (weights).

How can I fix a seed with stable diffusion 2.0?

Add LCM LoRA for Stable Diffusion XL

See https://huggingface.co/blog/lcm_lora.

Getting an error on the pipe.enable_xformers_memory_efficient_attention() step

Getting this error on the pipe.enable_xformers_memory_efficient_attention() step:

RuntimeError                              Traceback (most recent call last)
[<ipython-input-5-4f0f5a2b85ee>](https://localhost:8080/#) in <module>
     12     )
     13 pipe = pipe.to(device)
---> 14 pipe.enable_xformers_memory_efficient_attention()
     15 
     16 if model_id.endswith('-base'):

15 frames
[/usr/local/lib/python3.8/dist-packages/xformers/ops.py](https://localhost:8080/#) in no_such_operator(*args, **kwargs)
     44 def _get_xformers_operator(name: str):
     45     def no_such_operator(*args, **kwargs):
---> 46         raise RuntimeError(
     47             f"No such operator xformers::{name} - did you forget to build xformers with `python setup.py develop`?"
     48         )

RuntimeError: No such operator xformers::efficient_attention_forward_cutlass - did you forget to build xformers with `python setup.py develop`?

Any advice? Thanks

How can we load lora model(just a single file as .safetensors)?

How can we load lora model(just a single file as .safetensors)?

Switch model weights to stabilityai/stable-diffusion-2

Consider switching the model once stabilityai/stable-diffusion (demo and weights) is available via 🤗's D🧨iffusers.

Switch model weights to stabilityai/stable-diffusion-xl

Switch model weights to stabilityai/stable-diffusion-xl-base-1.0 (weights).
Maybe implement the second stage to refine the result with stabilityai/stable-diffusion-xl-refiner-1.0 (weights).