bbc-mc / sdweb-clip-changer Goto Github PK
View Code? Open in Web Editor NEWAdd setting to change CLIP. Extension for AUTOMATIC1111 web ui
Add setting to change CLIP. Extension for AUTOMATIC1111 web ui
@bbc-mc How to use sdweb-clip-changer in webui api
The current version (fecdf96) does not seem to work when installed on the forge extension.
I use both a1111 and forge and go back to a1111 when needed, but would like to use the clip changer in forge as well.
Hi,
Just wondering if this extension supports A1111 v1.6.x?
So far I have seen no sign of the extension downloading the clip I selected (openai/clip-vit-large-patch14-336).
I had to switch from the old one-click installer/Launcher to Stability Matrix, as the old one is no longer being developed for some reason, and I question whether or not Stability Matrix properly reports all CLI output, as it uses its own console rather than just opening a command prompt.
Would it be possible for you to add a "force CLIP model download" button, so that we can be sure the extension has actually fetched our selected CLIP model?
thanks in advance for any help.
I ran into some trouble using this with Counterfeit V3.0, which requires openai/clip-vit-large-patch14-336.
CLIPTextModel was on cpu
CLIPTextModel applied: openai/clip-vit-large-patch14-336
CLIPTokenizer applied: openai/clip-vit-large-patch14-336
Applying scaled dot product cross attention optimization.
Model loaded in 3.8s (load weights from disk: 0.1s, create model: 0.4s, apply weights to model: 0.4s, apply half(): 0.3s, load VAE: 0.2s, scripts callbacks: 2.3s).
Running on local URL: http://0.0.0.0:7860
To create a public link, set `share=True` in `launch()`.
CLIP Changer.on_app_started done.
Startup time: 8.7s (import torch: 0.8s, import gradio: 0.6s, import ldm: 0.4s, other imports: 0.4s, load scripts: 0.5s, load SD checkpoint: 3.9s, scripts before_ui_callback: 0.1s, create ui: 1.9s).
Error completing request
Arguments: ('task(ll4q9oh1fby7wn2)', '(masterwork, best quality, ultrarealistic:0.5). Energetic girl on the train', 'EasyNegativeV2', [], 20, 15, False, False, 1, 1, 7, 3870221604.0, -1.0, 0, 0, 0, False, 512, 768, False, 0.55, 2, 'Lanczos', 20, 0, 0, [], 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', 0, '', 0, '', True, False, False, False, 0) {}
Traceback (most recent call last):
File "/home/svein/AI/sd-bot/modules/call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "/home/svein/AI/sd-bot/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/home/svein/AI/sd-bot/modules/txt2img.py", line 56, in txt2img
processed = process_images(p)
File "/home/svein/AI/sd-bot/modules/processing.py", line 503, in process_images
res = process_images_inner(p)
File "/home/svein/AI/sd-bot/modules/processing.py", line 642, in process_images_inner
uc = get_conds_with_caching(prompt_parser.get_learned_conditioning, negative_prompts, p.steps, cached_uc)
File "/home/svein/AI/sd-bot/modules/processing.py", line 587, in get_conds_with_caching
cache[1] = function(shared.sd_model, required_prompts, steps)
File "/home/svein/AI/sd-bot/modules/prompt_parser.py", line 140, in get_learned_conditioning
conds = model.get_learned_conditioning(texts)
File "/home/svein/AI/sd-bot/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 669, in get_learned_conditioning
c = self.cond_stage_model(c)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/modules/sd_hijack_clip.py", line 229, in forward
z = self.process_tokens(tokens, multipliers)
File "/home/svein/AI/sd-bot/modules/sd_hijack_clip.py", line 254, in process_tokens
z = self.encode_with_transformers(tokens)
File "/home/svein/AI/sd-bot/modules/sd_hijack_clip.py", line 302, in encode_with_transformers
outputs = self.wrapped.transformer(input_ids=tokens, output_hidden_states=-opts.CLIP_stop_at_last_layers)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 816, in forward
return self.text_model(
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 712, in forward
hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 227, in forward
inputs_embeds = self.token_embedding(input_ids)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/modules/sd_hijack.py", line 234, in forward
inputs_embeds = self.wrapped(input_ids)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 162, in forward
return F.embedding(
File "/home/svein/AI/sd-bot/venv/lib/python3.10/site-packages/torch/nn/functional.py", line 2210, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)
...as you can see, the text model needs to be moved to the GPU.
However, like you can probably guess, I modified your code to print sd_model.cond_stage_model.transformer.device prior to swapping the text model. Which printed "cpu". Nevertheless, swapping to .to('cuda')
fixed the problem. I'm really not sure how that can happen -- maybe there's some code elsewhere in A1111 to move it, and swapping the model breaks that somehow?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.