Need help to fit free colab. about dreambooth-stable-diffusion

[tdins]

@robbsaber

1. Navigate to https://huggingface.co/CompVis/stable-diffusion-v-1-4-original
2. Click on “Access repository” on the left side of the page
3. If you already have an account, sign in — if not, create an account
4. When you’re logged in, navigate back to the original URL (the one I linked)
5. Click on ”sd-v1-4.ckpt” to download it
6. Once downloaded, rename the file “model.ckpt” and place it in your Google Drive root directory (resulting path: MyDrive/model.ckpt )

[TingTingin]

have you made progress on this?

[Mimocro]

@TingTingin, yep, have some of it. But even with much more black magic, tiny colab RAM is still a main issue. Other issues are colab specific and i think they are fixed now.
If just find the reason why trainer.fit() loads model to RAM, i think it should be working.

[Mimocro]

When i say load model from VRAM to RAM i mean model loads from cuda to cpu, It can be checked if crash model by setting precision to bf16 (google's cpus cannot work with bf16). After that, print(next(model.parameters()).is_cuda) returns False. Before that, trainer.fit print(next(model.parameters()).is_cuda) returns True.

Really have no idea.

[1blackbar]

so can you run it on pro p100 ?

[robbsaber]

Where do I find the models?
cp: cannot stat '/gdrive/MyDrive/model.ckpt': No such file or directory

[1blackbar]

I made a simple colab here https://colab.research.google.com/drive/1tugfQjRtH26QsX9lvMUdJqfCwWb5GHw4?usp=sharing With some black magic (such as loading state dict to 16gb vram instead 12gb ram, using tensorboard logger because new pytorch lightning cannot use testtube and so on) it can run all the code up to these lines:

  # run
  if opt.train:
      try:
          trainer.fit(model, data)
      except Exception:
          melk()
          raise

And i have 2 issues: First, TypeError: on_train_batch_start() missing 1 required positional argument: 'dataloader_idx' after trainer.fit(model, data), im sure what all dirs are correct and have no idea why. Also after that, model stores on cpu, not cuda. Second, on exeption, melk() do trainer.save_checkpoint(ckpt_path), and on this moment, RAM inflating very fast and reaches 12GB, so colab crashes.

It's just that i think running this in colab would be so cool, not everyone has 16GB of VRAM to load the model with full precision. Textual inversion is awesome too, but this one looks a bit more interesting.

Hey man, theres this repo that lets you run on 3090 with 24GB, so this should allow you to run on P100 32GB with colab pro, you think you can adapt the code so it would work ?
https://github.com/gammagec/Dreambooth-SD-optimized

[robbsaber]

@robbsaber

1. Navigate to https://huggingface.co/CompVis/stable-diffusion-v-1-4-original
2. Click on “Access repository” on the left side of the page
3. If you already have an account, sign in — if not, create an account
4. When you’re logged in, navigate back to the original URL (the one I linked)
5. Click on ”sd-v1-4.ckpt” to download it
6. Once downloaded, rename the file “model.ckpt” and place it in your Google Drive root directory (resulting path: MyDrive/model.ckpt )

Everything works now accept the last cells

IsADirectoryError: Caught IsADirectoryError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/gdrive/Shareddrives/G2/Dreambooth-Stable-Diffusion/ldm/data/personalized.py", line 188, in getitem
image = Image.open(self.image_paths[i % self.num_images])
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2953, in open
fp = builtins.open(filename, "rb")
IsADirectoryError: [Errno 21] Is a directory: '../Archive'

[Mimocro]

@1blackbar, i suppose, the limit RAM still the main problem. If colab pro with P100 can give more than 12.6 GB RAM, it is possible with replacing the git clone command (from this repo to optimized), and then running python main.py ... instead of last two cells. If cell stops and prints "^C", then It's still RAM limit.

[Mimocro]

@robbsaber, maybe, paths to something are incorrect?

[Mimocro]

@1blackbar, i suppose, the limit RAM still the main problem. If colab pro with P100 can give more than 12.6 GB RAM, it is possible with replacing the git clone command (from this repo to optimized), and then running python main.py ... instead of last two cells. If cell stops and prints "^C", then It's still RAM limit.

oh, i forget about one thing, testtube not working in new lightning, and old lightning not working in colab so that why i modified main.py...