isekaidev / stable.art Goto Github PK

Blurred mask's edges even with hard brush

The error happens when trying to inpaint. The resolution of the render was 400x600 px at a batch of 1 image. The Plugin spikes in vRam and then shoots out this error. The vRam then gets tied up and I have to close stable to dump it

Error code as follows

ERROR: Exception in ASGI application
Traceback (most recent call last):
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive
return self.receive_nowait()
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait
raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 77, in call_next
message = await recv_stream.receive()
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 114, in receive
raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 407, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 270, in call
await super().call(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 106, in call
response = await self.dispatch_func(request, call_next)
File "C:\ai\stable-diffusion-webui\extensions\auto-sd-paint-ext\backend\app.py", line 383, in app_encryption_middleware
res: StreamingResponse = await call_next(req)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 80, in call_next
raise app_exc
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 69, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 106, in call
response = await self.dispatch_func(request, call_next)
File "C:\ai\stable-diffusion-webui\modules\api\api.py", line 80, in log_and_time
res: Response = await call_next(req)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 80, in call_next
raise app_exc
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 69, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 26, in call
await self.app(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 706, in call
await route.handle(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 235, in app
raw_response = await run_endpoint_function(
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 163, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "C:\ai\stable-diffusion-webui\modules\api\api.py", line 230, in img2imgapi
processed = process_images(p)
File "C:\ai\stable-diffusion-webui\modules\processing.py", line 476, in process_images
res = process_images_inner(p)
File "C:\ai\stable-diffusion-webui\modules\processing.py", line 569, in process_images_inner
p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
File "C:\ai\stable-diffusion-webui\modules\processing.py", line 993, in init
self.init_latent = self.sd_model.get_first_stage_encoding(self.sd_model.encode_first_stage(image))
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "C:\ai\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 830, in encode_first_stage
return self.first_stage_model.encode(x)
File "C:\ai\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 83, in encode
h = self.encoder(x)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\ai\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 526, in forward
h = self.down[i_level].block[i_block](hs[-1], temb)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\ai\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 131, in forward
h = self.norm1(h)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\normalization.py", line 272, in forward
return F.group_norm(
File "C:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 2516, in group_norm
return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: CUDA out of memory. Tried to allocate 14.41 GiB (GPU 0; 24.00 GiB total capacity; 9.57 GiB already allocated; 11.88 GiB free; 9.58 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Hi, I was attempting to install the extension and kept failing to load with the error Failed to load Stable.art,
so I took a look at the code to try and figure out what the problem was and turns out the scripts were being loaded from https://stable.art/ not from the extension files .

<!doctype html><html><head><title>Stable.art</title><script defer="defer" onerror="handleScriptError()" src="https://stable.art/plugin-code/js/chunk-vendors.js"></script><script defer="defer" onerror="handleScriptError()" src="https://stable.art/plugin-code/js/app.js"></script></head><body><div id="app"><sp-heading size="L" style="text-align: center;" id="app-loader">Loading...</sp-heading></div><script>function handleScriptError() {
        document.getElementById('app-loader').textContent = 'Failed to load Stable.art';
      };</script></body></html>

so,why is it loading the js files from a remote site instead of the files that are bundled with it?

This was working a few days ago when I used the plugin last, but now the dropdown for model switching is greyed out.

I want to use my own, local instance of stable-diffusion with AUTOMATIC1111's webui as the backend, I've already set the --api parameter, but the plugin fails to load if I'm not connected to the internet. Why?

I'm assuming this is a bug since it wouldn't make sense to need to be online if I'm going to connect to a local instance (127.0.0.1:7860 being the default for AUTOMATIC1111's webui). Otherwise the plugin works as expected.

I'm using PS version 23.4.2 and after adding the plugin into the plugins folder, it does not show in the plugins menu. Does the plugin go somewhere else? I have tried unzipping the ccx file and doing a manual install as well.

When using a lasso selection that has been strongly feathered to do inpainting, the rectangle that is used for inpainting is too small and does not include the entire selected area.

Steps to Reproduce:

1. Make a lasso selection
2. Feather the selection by a significant amount (15px was used in my examples, on a 1920x1080 image)
3. Use the plugin to perform an inpaint using this selection, using a relatively high denoising strength to make the problem more obvious.
4. Observe that the inpainted region does not include the entire selection

Expected Results
The rectangle used for inpainting should include any pixels that are selected at all, even if the selection mask is mostly transparent for that pixel.

Actual Results
The selection mask seems to be getting alpha clipped, resulting in mostly transparent (deselected) pixels being ignored and the rectangle being too small. The visible result is that there are hard edges in the inpainted area.

Examples
A feathered lasso selection on the original image

A visual representation of that selection by filling it with solid white. Note the strong feathering and mostly transparent pixels that go even outside of the lasso "marching ants" graphic.

A couple of inpainting results using the above selection. Higher denoising strengths make it more obvious. Notice the hard edges of the rectangle that fall well within the selection.

It would be nice to have the option to have the plugin only attempt to connect to the webui API when I manually tell it to. Such as a "Connect" button under the Endpoint text box.

I use Photoshop frequently for things other than just stable diffusion, but if I open Photoshop while the webui is not running, I get a disruptive error telling me that it failed to connect. I would prefer if it would not even try connecting unless I tell it to.
Perhaps it could have a checkbox for automatically connecting so that users who only use Photoshop for this plugin can continue using it like before.

When using very long prompts, it becomes annoying to navigate the text boxes for them as there is no scroll bar. The only way to scroll is to use the arrow keys to move the text cursor.

When trying to generate an image I get this error : "ReferenceError: Abort controller is not defined"

Hello there! I got problem when i'm trying to use inpaint, i got an error 500 in photoshop and in cmd i saw that program can't find ESRGAN_4x! I　download this upscaler to models but this doesn't help me.. Also i have Stable diffusion 1.5 maybe problem in this i don't know )Thanks you in advance! Ectually best extention for Photoshop ever!

Hi! I got this problem on my mac and cannot fix it much.

I grant access to "Accessibility" because i saw that someone just reseted firewall. I don't have firewall so i don't think that's the problem. Can you please help?

Lasso the selection

Click inpainting then hit generate to get these errors:

PS 23.3.2

I can't seem to make img2img mode work, see video:

using PS 23.3.2

I have Photoshop 24.1.1 installed yet this error comes up when following the plugin installation instructions. What could cause it?

Is it because of uxp or some special support in new version

PS Version: 24.0.1

Sometimes two different images in a batch can both have different elements that they do better than each other, and the ideal result is to accept them both and blend them together to get the best combination. Currently doing this requires manually copying and pasting a generation and then accepting another one. It would be very nice to have some option to choose more than one generated image from a batch to have them all added as smart objects. Perhaps a checkbox or shift+clicking to multi select.

A button that, when clicked, saves a temporary flattened copy of the current document and sends it to the img2img tab in the AUTOMATIC1111 webui so that more advanced img2img operations can be performed on it.

I know that Stable Diffusion doesn't know yet what alpha chanel is and can't make transparency. But considering that it's a photoshop plugin, maybe it can now? So is it possible to generate with transparent background using this plugin? Or maybe will be possible?

I have a few suggestions for improving the Stable Diffusion addon:

1. When the app crashes, a prompt appears that won't close no matter how many times you click “okay.” This can be annoying because it requires you to end task all of Photoshop. I found a way to keep Photoshop open, but I can’t guarantee that this will always work. I suggest finding a way to fix this issue so that it doesn’t cause any further problems, especially when you’re in the middle of a project.

2. It would be helpful to have a checkbox or something similar that allows users to save their prompts for next time. This way, they can resume their work even after a crash. This checkbox should be able to be turned on or off, so users can secure their privacy if they wish.

3. Off topic to the crash, but it would be great to implement controlnet in the app, at least for depth map and canny.

4. Add codeformer to upscale faces.

Overall, this is an excellent app and I really appreciate the developer for their contribution to the evolution of digital imaging.

Hi. I’m so sorry. I’m completely lost, and have no idea what I’m doing. I’m just trying to get the Stable.art Photoshop plugin to work, but whenever I click Generate, it gives me the following error:
Error: NAPI API failure: Number expected

I followed the instructions here:
https://github.com/isekaidev/stable.art#how-to-install
But I got stuck at:
https://github.com/isekaidev/stable.art#start-api-server
I use Automatic1111 locally, so it said to follow the guide here:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API
Which I tried, and when I open http://127.0.0.1:7860/docs, it does look very similar to the images they have there, but the moment they started talking about “constructing a payload with the parameters”, I have absolutely no idea what they were talking about. I posted in their discussion, but they said that if I have --api in my commandline args, and I see the API endpoints when going to /docs, that's pretty much it on the Web UI side.

I'm not sure what else to say about the error.

My Stable Diffusion Automatic1111 runs fine.
I run the with --api after set COMMANDLINE_ARGS= in webui-user.bat
I launch Photoshop
The plugin seems to load fine in Photoshop
I make a selection with the Rectangular Marquee Tool
I put the local URL in the Endpoint box: http://127.0.0.1:7860
The Model option has default (current webui value) checked
I put any Prompt in
Negative Prompt empty
Seed empty (or a number like: 1333168849)
Sampling method: DPM2 Karras
Steps: 20
CFG Scale: 7
Mode: txt2img
Advanced Settings: Number of images: 4

I click Generate, and it gives me the error message. Generating sits at (0%) indefinitely (unless I click Interrupt,) and sometimes it scrolls back up to Seed, which led me to believe it had something to do with the Seed, but as I said, I've tried leaving it empty, and I've tried putting in a legitimate number like 1333168849 (copied from a successful generation in the webui,) but it's still the same result. I can change models just fine, but that's it.

Again, I'm not sure what else to include.

So if anyone could let me know what I’m doing wrong, or point me in the right direction, I’d certainly appreciate it. Thank you in advance.

          It means that you launched your webui but did not enable API. Try to open {your_url}/sdapi/v1/sd-models (e.g. http://127.0.0.1:7860/sdapi/v1/sd-models), if it does not open, then you did not enable API. Check this guide on how to enable it https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API

Originally posted by @isekaidev in #24 (comment)

txt2img works without any problems.

But img2img and inpaint don't work.
I click Generate and Photoshop tries to generate the mask
When an error occurs:

Cannot export stableart_exported_mask.png.
Try to reload Photoshop and check that "Quick Export as PNG" is working

After confirming the error, another appears:

Error: Cannot export
stableart_exported_mask.png

Photoshop: 24.1.1

my server is listening the port 80 and can be accessed from my laptop;

but on the same laptop, in PS and stable.art, it just can't connect to it.

any suggestions?

Look like it generates imgs with 512x side resolution and resizes it to a desired selection when i trying to img2img/inpaint for example ~800x1000

Hello, this is a great plug in, but from my testing, generated images aren't saved in the automatic1111 output folder.
I would very much like this option because it's easy to come back to those output images and read out the prompt and model information.

Of course, it would be even better if the Photoshop UI had access to the automatic1111 output folder(just like the 'explore' tab, but for the local automatic1111 output folder.) the so that previously generated images could be used. But for now, I'll just be happy if all generated images are saved in the automatic1111 folder.

(It would be even better if a specific subfolder would be created for images generated using the plugin in the automatic1111 output folder)

Inpainting is probably the most powerful use of Photoshop + Stable Diffusion. However, without mask + mask blur, many of the generated images won't blend nicely to the rest of the image.

It would be great if you could add this feature (or at least a mask blur automatically on the border of the selection) to your plugin.
Thank you for your creation!

An option in the UI to directly set what pixel resolution the generation should be generated at. If this plugin is used on relatively large documents (2000px is not uncommon for digital painting) with a standard GPU such as a 3080, it runs out of memory. The only solution is to inpaint smaller sections or manually scale the image down before performing inference.

I think the ideal solution would be to have an optional setting (Maybe under a collapsed "Advanced" section?) for manually setting what resolution it will be generated at.

This is a great tool because it's not bloated.
If you want to add lots of features, consider keeping this version also.
Or having the option to not upgrade to the future version with so many drop downs and options.
Thank you again for your fantastic work.

Currently, when the prompt text box is focused and Ctrl+Z is pressed, it works like pressing Ctrl+Z in photoshop itself does, flipflops between undo and redo. However, pressing Ctrl+Shift+Z does not behave the same way that it does in photoshop itself, and still just flipflops between undo/redo rather than stepping back in an undo history.

I think this is not ideal behavior and there are two possibilities for how it should behave instead:

1. Ctrl+Shift+Z should step back in undo history like it does in photoshop itself. I understand this may be a big ask if there isn't already an implementation of an undo history for prompt entry.
2. The text box should ignore Ctrl+Z and Ctrl+Shift+Z keyboard shortcuts. When the user uses those shortcuts while the text box is in focus, photoshop just behaves as normal and the text box is unchanged.

why this is happening?

The text boxes for entering prompts in the plugin panel have strange behavior with a couple of keyboard inputs that I use often. They are: Ctrl+Backspace and End.
The former inserts a special character rather than backspacing a whole word like it does in standard text entry boxes.
The latter scrolls to the bottom of the plugin panel rather than moving the text cursor to the end of the line.

Hello. I have an issue.
My Server flow generate, process going after 5-10% a have a error. In server-side: status goes to 100% but images on server-side won't saved.

In addition to the dropdown for choosing models, it would be very helpful to have a matching dropdown for switching Hypernetworks that the AUTOMATIC1111 webui is aware of. The same goes for VAEs.

Hi there,

First of all I wanted to thank you for your hard work and sharing this project. In my opinion, it is the best Photoshop SD project I've seen so far. The API connection works great, allowing easy access to different models. Being able to make selections instead of masks also works way better than I initially expected. And that for a project which had its first release last week!
Like your username, it really feels like you and Stable.Art are from another world :)

Issue wise there aren't any major "bugs/complaints" at all which surprised me after testing out a lot if not most of the other Photoshop SD projects. The minor bugs like variation in color are easily fixed with a hue/saturation mask, rough edges with a mask and a soft brush and if I remember correctly, the README said you are already working on outpainting.

I'm never going back to another Photoshop SD plugin ever again :)

When using a non-rectangular selection to perform inpainting, such as a lasso selection, the resulting Smart Object should come with a layer mask that masks out the parts that were not included in the selection. This would be especially useful when using more advanced selections such as a feathered lasso selection.

Stable Horde is a FOSS crowdsourced cluster of stable diffusion workers. It provides a free to use and fully documented REST API. Integrating it would allow your users to use this plugin without a GPU or the technical know-how to install it.

so far so good! im loving the simplicity of the plugin.

When inpainting the results display at the wrong scale. If i select a 200x200 pixel mask, when displaying the renders, it scales them up almost 400% from the top left point of the mask

Sometimes i like to have all of my generated results available for later use. Is it possible to set it up so that it saves a local copy of the files somewhere? Also, where is the temporary folder used to store the generated results?

Simply adding the "styles" Feature from the auto1111 UI to the prompt section to allow for simple and quick prompt presets

When prompt or negative prompt is very long it bugs the input box. You can't see the prompt, you can't change anything in it, when you push it, it just make outline blue then nothing. Adress of A1111 is entered, I just erased it for secure purposes.

This is just a suggestion, but if you could implement ControlNet in the plugin, it would be amazing!

https://github.com/lllyasviel/ControlNet

Add inpaint at full resolution with option to choose desired resolution.
Also need an option to choose type of masked content like in A1

Hi there,

I am having an issue that I cannot seem to resolved. These are some of the things that I've done and I still get the same error.

• Reinstalled the latest version of Adobe Photoshop.
• Disable all firewall app.
• Tried to load Photoshop from new windows admin user.
• Load Photoshop as Admin.
• Load Automatic1111 before starting Photoshop.
• Adding, removing, reinstalling Stable.art, redownload CCX and tried again.

I've installed this at my work computer and it has no problem, but something on my home computer is causing a problem. Appreciate your assistance.

I can't get the plugin to work.. when I put the endpoint URL.. I get this error message whether it's local or remote on another machine:
Error: cannot connect to your API

Hey there, I run stable diffusion locally without any issue usually, however this plugin will automatically create images in batches of 4, which increases memory cost and causes me to get CUDA out of memory issues much more often.

Would it be possible to get a slider for batch count added in, along with one for image resolution?

This plugin has the most potential I've seen to become perfect - thank you so much!