Feature Request: 3rd Party AI Background Remover about photopea

[Devetec]

I like the idea of it being a feature for Premium users, but one of the nice things about Photopea is that every feature is available to everyone (at least, afaik), not just Premium users.

[camoody1]

I like the idea of it being a feature for Premium users, but one of the nice things about Photopea is that every feature is available to everyone (at least, afaik), not just Premium users.

I agree... in general. Unfortunately, when a feature will actually cost the developer an ongoing monthly fee, I could absolutely see locking it behind a premium paywall. It's not just a case of Ivan creating the code once and the cost being done. It would be an ongoing cost. Maybe he could bundle it with some other AI capability that costs him money. I mean, keep in mind, we're talking about just $3.33 per month for an absolutely amazing product that is constantly being updated. I wish he had his own server farm to do all these new AI functions for users, but that's just not the case. 🤷🏽‍♂️

[Devetec]

I was thinking more and I think that it would be possible to make this with a plugin (which I discovered are a thing today!)

There already is some other AI capability, that being DALL-E & Stable Diffusion, but it is heavily rate-limited for users unless they use their own API keys. I think if Stable Diffusion is able to be a feature within Photopea, so can background removal and other types of things like that.

[photopea]

I am very surprised, that these services are offered quite cheaply. I definitely want to add it to Photopea, and see how many people use it. If there are too many people, I would limit it to 5 images per hour, or Premium only, etc.

The only problem I have ... how does it compare to www.remove.bg ? Could you do a little research and check, if the results are at least as good as from remove.bg ? Or show us a few examples?

[camoody1]

I am very surprised, that these services are offered quite cheaply. I definitely want to add it to Photopea, and see how many people use it. If there are too many people, I would limit it to 5 images per hour, or Premium only, etc.

The only problem I have ... how does it compare to www.remove.bg ? Could you do a little research and check, if the results are at least as good as from remove.bg ? Or show us a few examples?

I was thinking more and I think that it would be possible to make this with a plugin (which I discovered are a thing today!)

There already is some other AI capability, that being DALL-E & Stable Diffusion, but it is heavily rate-limited for users unless they use their own API keys. I think if Stable Diffusion is able to be a feature within Photopea, so can background removal and other types of things like that.

Yes, this is absolutely going to be a plugin if Ivan wishes to implement it. The entire process is an API call where he sends the image to their servers, they process the image, and then return the result. Each image processed costs a tiny amount of money, but when you scale that up to a 400,000 daily user base, the money could potentially get out of hand depending on his budget. That's the whole reason he hasn't already done something like this in the past.

[camoody1]

The website I linked is a demo page that allows you to upload any image under 16MB for testing. Unfortunately, they don't allow you to download the results. This is strictly an API based tool, not made for the everyday user to use from a website gui. So, the best I can do is upload some images and then show you the screenshot results. Again, here is the demo page if you want to test for yourself: https://api4.ai/apis/bg-removal

[camoody1]

[camoody1]

[camoody1]

[camoody1]

I've attached some samples. Once you've seen them for quality, please delete them. 😁

[camoody1]

Here is the API information page showing the syntax for sending images and receiving the results.
https://rapidapi.com/api4ai-api4ai-default/api/background-removal4/details

I'm not a programmer, but if I'm reading this correctly, I believe one of the output options is for them to return a mask to cover the background and show the subject, instead of returning JUST the subject. This option would be great for allowing users to cleanup any mistakes the API made.

[Devetec]

I think there might be higher-quality background removal APIs, for example, for remove.bg here's an example (using one of your photos, didn't save it to my computer and also delete this one after seeing the quality!)

[Devetec]

[camoody1]

Keep in mind, as I stated, what I uploaded were screenshots from my phone, so they appear a bit lower quality than your download. But yes, the cutout from remove.bg DOES look a little bit better than what I uploaded. Two very important things, though... First, from what Ivan has said, the remove.bg pricing is orders of magnitude more expensive per image than the one I have suggested. Budget is a real limitation in this decision, I would assume. And second, the API I'm suggesting will allow for masks to be sent back as the result allowing for easy cleanup of any spots that aren't perfect. I wouldn't normally suggest settling for a slightly lesser process, but when the pricing is THIS much better, I'd have to give it a real look. Sadly, the pricing of the remove.bg API makes it a non-starter.

[camoody1]

By the way, this is the result remove.bg returned for the picture of the boy standing in the closet. So, it looks like the two APIs are basically equal on average. It's just that remove.bg costs 10x more per image.

[photopea]

I have added this option into Select - "Remove BG". It will create a raster mask on a current layer.

I have paid for 50K images a month. Hope it will not be used in the next two hours :D

[camoody1]

Ivan, the work you do on this application is absolutely outstanding. I'm sure people will use this option a LOT once they first find it, but hopefully it will settle down and only be used at a normal rate once it stops being the "new" thing. This capability is absolutely a huge step up for selecting subjects in Photopea.

[camoody1]

I sure hope the 4096x4096 limitation doesn't become too much of a problem for people. I checked the API information for remove.bg and they only offer PNG results up to 4000x2500, so it doesn't look like our option is especially limiting. 🤷🏽‍♂️

[camoody1]

I wonder if you could scale down the image before passing it to the API, and then scale up JUST the mask once it is sent back and apply that scaled up mask to the original image? This would allow you to stay under the API limit, but still let the user keep their super hi-res image.

[photopea]

This feature can process images of any size.

[camoody1]

[photopea]

Thank you guys :)

I am thinking about a good text-to-image generation service. Currently, we use DALL-E from www.openai.com (under Window - More - Stable Diffusion). But it is quite expensive (2 cents per image).

[camoody1]

Thank you guys :)

I am thinking about a good text-to-image generation service. Currently, we use DALL-E from www.openai.com (under Window - More - Stable Diffusion). But it is quite expensive (2 cents per image).

Yeah, I've been researching different options for inexpensive text-to-image options and haven't really found a good one that isn't so expensive. If I find one, I'll surely let you know.

I think a more important option for Photopea to have in the AI area before text-to-image would be an Image Enhancer (to remove blurs) or an Image Upscaler. I think those two AI options are more in line with what Photopea is as a tool before bringing DALL-E type functionality into the tool. Although, an AI option for doing Outpainting to fill-out a smaller image would also be a great tool that fits well into the Photopea environment.

[photopea]

I have found this so far: https://rapidapi.com/dezgo/api/dezgo/pricing . Processing one image costs 0.0001 USD, which is 200x cheaper than DALL-E from www.openai.com But I am not sure about the quality.

[camoody1]

I have found this so far: https://rapidapi.com/dezgo/api/dezgo/pricing . Processing one image costs 0.0001 USD, which is 200x cheaper than DALL-E from www.openai.com But I am not sure about the quality.

Check that pricing, again. It costs $0.0001 USD per CREDIT. Each image costs a certain number of credits depending on the size of the image and the process being done. For instance, each 1024x1024 image costs 181 credits to complete, or $0.0181 USD. A 512x512 image costs 19 credits, or $0.0019 USD. They also offer an "upscale" option that would reduce the cost of that 1024x1024 image by actually creating a 512x512 image and then upscaling it x2. The price of that option would be 19 credits for the image plus an additional 12 credits for the upscale for a total of 31 credits, or $0.0031 USD.

So, the pricing COULD be cheaper than your current option depending on the sizes of images passed and processing done. But it won't be on the order of 200x cheaper. At most, it would be closer to 7x cheaper. Of course, that's still a nice money savings. 🤷🏽‍♂️

Sorry to bring bad news.

Having said that, the OPTIONS this API gives you are far superior to what your current DALL-E functionality brings. Allowing the user to choose the model to be used is a really nice upgrade, as is the image-to-image option.

I still tend to think it's probably a better idea to let users do their own text-to-image work outside of Photopea, or to find a good tool and design a UI for them inside Photopea, but make them use their own API key. You could choose a tool that has basic options for free API keys and more advanced options for paid keys. That would save you the cost of people experimenting constantly with images. I'm not sure how much you've played around with text-to-image yourself, but it's really easy to create 20 to 30 images in one sitting and that would get really expensive if you scale that up to your entire user base.

I think your next focus for AI in Photopea should be AI Upscaling of images, AI Enhancement of blurred images, and (possibly) AI Outpainting of images to allow for improved canvas enlargement. Those sort of processes, I think, are more inherent to the core of what Photopea is meant to be as a tool. Photopea is for designing, manipulating and fixing images and photos... not for the repetitive try/try again nature of the current text-to-image scene. Other tools made exclusively for that type of work will always do it better than what you can because Photopea wasn't designed to be that sort of tool.

Just my opinion.

[photopea]

I did not notice credits. So it is not 200x cheaper, but still 7x cheaper.

These text-to-image tools can often do inpainting / outpainting, so I guess it is pretty much the same thing.

There was a huge hype of people wanting text-to-image in Photopea, just because everyone was talking about it, and other editors offered it. But I agree that it is not that useful in practice, you just play for a while until you get bored: #5000

[MichalSZZ]

There was a huge hype of people wanting text-to-image in Photopea, just because everyone was talking about it...

For me, very useful in daily work are:

• content-aware fill (AI inpainting method is excellent)
• better selecting objects (this is most wanted by the user - precise selecting and background removal)
• upscaling images

but you can think about giving an option to generate text-to-image for free (with a limitation for registered users) as a another way to promote Photopea

[camoody1]

There was a huge hype of people wanting text-to-image in Photopea, just because everyone was talking about it...

For me, very useful in daily work are:

• content-aware fill (AI inpainting method is excellent)
• better selecting objects (this is most wanted by the user - precise selecting and background removal)
• upscaling images

but you can think about giving an option to generate text-to-image for free (with a limitation for registered users) as a another way to promote Photopea

Background removal is now using AI in Photopea and it works really, really well. Ivan implemented that late last week. 🔥

[camoody1]

I did not notice credits. So it is not 200x cheaper, but still 7x cheaper.

These text-to-image tools can often do inpainting / outpainting, so I guess it is pretty much the same thing.

There was a huge hype of people wanting text-to-image in Photopea, just because everyone was talking about it, and other editors offered it. But I agree that it is not that useful in practice, you just play for a while until you get bored: #5000

@photopea It's only been an hour, and I'm already slightly rethinking my opinion. 😆

While I still definitely believe the bulk of any AI Images creation should be done outside of Photopea, I would ammend my original thought to to say that in addition to AI Image Upscaling, AI Image Enhance (for blurs) and AI Image Outpainting (to enlarge the canvas), I can definitely see where AI Image INPAINTING would be super beneficial, as well. And I do believe Inpainting still falls under the general purpose core of Photopea. Small additions to an overall image or photo is different than designing an entire image from scratch and trying over and over until you get something you like. And to prevent people from wanting to abuse it, I wouldn't even label it as "DALL-E" or "StableDiffusion". You could just put it under filters as "Inpainting" or something like that. The only issue would be how you would handle passing a large image (say 4096x4096) thru the API to get a small inpainted image (of say 512x512). Obviously, you'd only want the small inpainted image returned, but you would have to pass the full larger image into the API for the whole image to be processed. I'm afraid that reducing a larger image down to 512x512 for processing and then blowing it back up on return up would give us in some really poor results. But maybe you know if some magic for that.🤷🏽‍♂️

[photopea]

I have implemented the Stable Diffusion under Window - More - Text to Image.

It is from Dezgo: https://rapidapi.com/dezgo/api/dezgo/pricing and it is 10x cheaper than DALL-E. DALL-E is availabe only with your own key now.

[photopea]

Ok so I will close it now. BTW. or service supplier said that they are planning to improve the background remover even more, so it should give better results soon.

[camoody1]

Outstanding. It already gives really nice results. Glad I found it and that you were able to do quickly implement it. Nice win for the app.

[camoody1]

@photopea Dezgo just released a new background removal tool that will go as large as 2048x2048 through their API. The cost is $0.0010 per removal and you can choose to have it return a mask like you currently do. I think this is slightly cheaper than your current provider. I thought you might want to chat with Clem to see what they can do for you.

https://dezgo.com/remove-background

[photopea]

Man, is this even real? The results of https://dezgo.com/remove-background are absolutely unbelievable! Here is our current removal in the middle, Dezgo on the right side:

[photopea]

I have implemented it! @camoody1 Thanks a lot for telling me about it! It is a bit more expensive than our current background removal, but I think the result is worth it. Just look at this, omg :D

image.mp4

[camoody1]

@photopea Wow! That really is a nice difference. The original tool was developed many months ago - which is like a lifetime in AI development. I'm sure Dezgo is using a much more powerful implementation.

You should consider checking out a project I've been following on GitHub by a guy who has written a plugin for Photopea that connects it to Stable Diffusion image creation through an Automatic1111 backend. It's a very advanced implementation. It's not something the average user can take advantage of due to the processing power and disk space requirements, but the tech is there. Perhaps you could reach out to the developer and see if he would write something to take deeper advantage of the offerings that Dezgo has? Perhaps he could help you develop a true "Generative Fill" competitor?

If you care to take a look, here is the GitHub page:
https://github.com/huchenlei/stable-diffusion-ps-pea

[photopea]

@camoody1 I think this project is for those who have a stable diffusion installed on their device. It is quite complicated to install, and you need an expensive hardware to run it at a reasonable speed. I dont think we should promote this plugin to "masses" and ask our users to install Stable Diffusion. The idea of Photopea has always been "open a website and use it right away".

[camoody1]

@photopea Yessir. I am aware of what the plugin is. I've been using it off and on. I have Stable Diffusion on my PC... though it runs very slowly on my old GPU.

The plugin works quite a bit with inpainting and outpainting, so I thought maybe he would be able to use that knowledge to create a separate plugin that uses Dezgo API calls as the backend instead of Stable Diffusion. He's obviously already very familiar with models, configuration settings, steps and resolutions... all of which are needed both with Stable Diffusion and Dezgo API calls. He could potentially develop a plugin that would allow users to have their own Dezgo login and use their own money for the API calls. It's super cheap for an individual user, but far too expensive for you to pay on your own for everyone as the developer.

[photopea]

I have noticed, that many cases of the background removal can be done with the Magic Wand. Now, Photopea detects these cases and removes the background without sending it to Dezgo :) You can try it for images below.

[camoody1]

@photopea Oh, this is smart. Images with solid blocks of color like this in the background are perfect for the Magic Wand. That a nice bit of smart savings. Well done.