bbc-mc / sdweb-clip-changer

Add setting to change CLIP. Extension for AUTOMATIC1111 web ui

CLIP changer

日本語READMEはこちら

• This is Extension for AUTOMATIC1111's Stable Diffusion Web UI
• Add feature to change CLIPTextModel and CLIPTokenizer

Update

• 2023/06/06: fix: add support for lowvram/midvram

  • Fix problem about tensor should load on same device when use option lowvram midvram.

How to install

• Open Extensions on webui

• Install from URL

• Install

How to use

• Open "settings"
• Input CLIP address for your needs

Note

This extension will exchange CLIP at "after model Loaded".

It means,

• no change applied until model re-loaded if you change setting

• also no change applied until model re-loaded if you Disable this extension

You should care about which CLIP is now applied.

This function may cause problem with model merge / training.

Settings

In "Setting" tab	Sample
Enable CLIP Changer
Specify CLIPTextModel to use.	openai/clip-vit-large-patch14
Specify CLIPTokenizer to use.	openai/clip-vit-large-patch14

Supported CLIPTextModel, CLIPTokenizer is diffusers

from transformers import CLIPTextModel, CLIPTokenizer

Sample of settings

• Run test with,

  • openai/clip-vit-large-patch14
  • openai/clip-vit-large-patch14-336

• sample image

Special Thanks

• 「[WebUI] Stable DiffusionベースモデルのCLIPの重みを良いやつに変更する」