nonnonstop / sd-webui-3d-open-pose-editor Goto Github PK

It would be helpful to have custom buttons for navigation. For example, middle mouse button for moving scene, right mouse button for rotating scene.
Thanks!

For example, setting the OpenPose map to use the OpenPose model or the depth map of the hands to use the depth model in the settings automatically so it doesn't need to be set manually.

Just a QoL feature that saves a bit of time.

I always use mmd to posing.
I hope it can import .vpd file

ValueError: File cannot be fetched: /content/drive/MyDrive/SD/extensions/sd-webui-3d-open-pose-editor/style.css. All files must contained within the Gradio python app working directory, or be a temp file created by the Gradio python app.

tunnel : ngrok

After an update, the plugin completely stopped working. There's no bones. So it renders the extension not usable.

Steps to reproduce the behavior:
Go to go to the extension. Shift+D for duplicate bones/ new bones.

Expected behavior:
Bones will appear.

• OSX 13.3 Ventura, MacBook Pro M1
• Browser: Brave
• A1111 WebUI Commit: 22bcc7be
• Torch: 2.1.0.dev20230401
• Extension: 6004a31 (Sun Apr 9 05:42:42 2023)

After updating the extension, I get this error:

API error: GET: http://127.0.0.1:7860/file=undefined {'error': 'RuntimeError', 'detail': '', 'body': '', 'errors': 'File at path D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\undefined does not exist.'}
Traceback (most recent call last):
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive
return self.receive_nowait()
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait
raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\modules\api\api.py", line 145, in exception_handling
return await call_next(request)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 84, in call_next
raise app_exc
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 70, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 108, in call
response = await self.dispatch_func(request, call_next)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\modules\api\api.py", line 110, in log_and_time
res: Response = await call_next(req)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 84, in call_next
raise app_exc
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 70, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in call
await responder(scope, receive, send)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in call
await self.app(scope, receive, self.send_with_gzip)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 718, in call
await route.handle(scope, receive, send)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 69, in app
await response(scope, receive, send)
File "D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 338, in call
raise RuntimeError(f"File at path {self.path} does not exist.")
RuntimeError: File at path D:\DEVELOPMENT\STABLEDIFFUSION\stable-diffusion-webui\undefined does not exist.

Stable Diffusion works normally and no errors if I disable this extension.

You should have option to use sticks as bones cause theyre too large when you do head close up and openpose wont recognize skeleton at all on close up of head, try it
Bones should be that thin

Hi there. Thanks a lot for this great extension! Two things that would come in really handy ^^

1. Reset Button: ok I see restore last scene is like the reset to the original, if no other scene is loaded :)

2. Hands are not easy to configure with this model (at least not for beginners). It would really help and safe some time, to have hand gestures like "fist" "stretched out hands" some of the most common hand signs be "loadable" and be able to adjust from there.

Thanks again, maybe sometime it can be implemented :)

Greets

clean install of the extension results in:

RuntimeError: File at path extensions/sd-webui-3d-open-pose-editor/models/shou.jpg does not exist.

looking at the files, it seems to be referenced by models/hand.fbx model so model loader is failing.

Not doing all the suggested noise because I don't have time for that at the moment. Saw a similar issue, BUT mine is slightly different. I have a white screen, with the UI, but no skeleton. I can technically generate and send the skeleton out of the editor, but i have no way of seeing what I am doing.

I've installed the extension. The Edit Openpose tab works fine. After clicking on the Create button it switches to the Send to ControlNet tab. I can see the four images are populated. But when I click on those two Send buttons nothing happens. I've tried it on Chrome and Firefox.

What have I done wrong?

It worked before but now it's broken. I don't know what affected it.

TypeError: Cannot read properties of undefined (reading 'map') at i2.DetectFromImage (http://127.0.0.1:7860/file=extensions/sd-webui-3d-open-pose-editor/javascript/lazy/main.js?c7969a7a:28054:40)

Clicking the function button does not respond, and ControlNet does not receive the corresponding picture

I've uninstalled and reinstalled the extension and it's no longer showing the most recent updates to the UI. Keeps switching back to the original version without the Gradio controls.

OR at least upload an image with openpose skeleton sticks and it would try to match it in 3d view, its so cumbersome to move skeleton around and hands are literally impossible to get right, so a library for hand poses would help a lot
Separate lib for hands and separate for poses
Also default focal should be 100mm , SD isnt great with perspective

TypeError: Cannot read properties of undefined (reading 'map') at i2.DetectFromImage (http://127.0.0.1:7860/file=R:/stable-diffusion-webui/extensions/sd-webui-3d-open-pose-editor/javascript/lazy/main.js:28054:40)

Some people reported that they couldn't click any joints, I fixed the online version. Please merge the following commits.
ZhUyU1997@83d0404

Thank you for making this Extension.
The camera and bone system give AI art a new sense of certainty.

Could you implement additional functions to make this function even easier to use?

"Detect from image" would be very useful to have a function to estimate the pose taken by the character in the loaded background image.

Since the functions are only pseudo-inferences, we think that the pose does not need to be accurate in the world space as long as it can infer the state seen from the current camera.

Describe the bug
updated to 6a047e8 (Sun Apr 2 10:50:58 2023)
click on 3d openpose tab and it all looks mangled inside a tiny window, can't use it

To Reproduce
Steps to reproduce the behavior:

1. Click on 3d openpose tab
2. Edit Openpose looks very small

Expected behavior
Be able to use the UI

Screenshots

Environment (please complete the following information):

• Windows 10
• Edge and Firefox
• A1111 WebUI Version: Commit hash: 64b7e8382377bb578d9740c061979776b214cfd9
• Extension Version: 6a047e8 (Sun Apr 2 10:50:58 2023)
• ControlNet Version: 241c05f8 (Thu Mar 23 15:18:35 2023)

Additional context
maybe this is some simple css html dimensioning problem. The mini window looks to have stuff in it.
The "Send to ControlNet tab is perfectly normal looking and usable, it's the Edit Openpose one that is diminished.

The 3d open pose editor is white or blank in opera gx, the extension is working properly on chrome

This possibly has something to do with the issue

In the title. It would be really cool if we could open saved JSON files generated from OpenPose Editor. Then we could fine tune hands and poses in general. This would be awesome. :)

Your making is great, but I think you should add existing openpose setting, because the Detect From Image Function doens't work well, espacially when some parts of the body are not on the photo.

Thank you, @nonnonstop

ps. I think you should also add turning camera that can be seen leaning.

Something went wrong! TypeError: Cannot read properties of undefined (reading 'map') at Av.DetectFromImage (http://127.0.0.1:7860/file=I:/stable-diffusion-webui/extensions/sd-webui-3d-open-pose-editor/javascript/openpose.umd.js?1679236453.501455:3398:17542)

I want to insert an iframe linking to the online version.
The iframe communicates with the webui via postMessage.

1. Users of the extension can easily get the latest changes.
2. Webui upgrade will not break the editor

I am a beginner.
I have completed the installation. but I can't find a "body parometers" diagram.
It's very troubled.
How can I fix this?

The default settings width/height= 512. After creating the pose, I clicked on Download. The image is saved as 640x640.

The barefoot model sometimes Makes AI generated foot like hand.
I guess it was because the shape of toes makes AI think of fingers.

To fix this, I suggest to have multiple foot model for users to swap.
Since only an outline is needed, we don`t need to have tons of shoe model for our case.
I believe the following model would be enough, priority high to low.

• Flat shoes, general design, fits most of the cases.
• stiletto heels, a design with long, thin heels, this will cover most of the heeled shoes such as pumps, heeled boots, etc.
• wedge heels, a design with no gap between sole and heel, covers the rest of heeled design.
• Paws, for furry lovers (perhaps a scale option might be needed for optimal outcome)

to bones for these shoes (exclude paw) could be simple since toes are not there.

Thanks :)

it will generate them in the send to text/img panels, but not on the main page where I can adjust everything

like the work so far, but setting a pose using radials to rotate each bone is cumbersome - great for fine tuning, but not that great to set the general pose.

suggestion is to have free-move mode where click & drag moves a bone on two prominent axes and lock the third axis
for example:

• if using front view, it mouse moves on x/y and z is locked
• if using side view mouse moves on y/z and x is locked

and determining which two axes are prominent can be determined from camera angle

second part of feature request is to actually utilize right click to add a popup menu - delete/duplicate/move-mode - they can all be embedded controls so main ui can be vastly simplified.

and entire camera params section can be removed - the full functionality is via mouse movement anyhow.

1. Control panel is covering the preview. How can I shift them?
2. After Fixing View. Restore View doesn't seem to work.
3. Is there a way to restore skeleton back to default pose? e.g. after I messed it up or duplicating one and want to start from scratch.
4. I always have to hold X key to move skeleton and can't switch to move mode like the old version. That was useful when I have several skeletons to position. I could click and move.

Thanks for your amazing work 👍

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
当我在使用从图片检测姿势这个功能的时候，会有一段时间用来下载media pipe模型，对于**境内的用户，这种下载似乎因为网络原因无法进行，导致一直停在下载页面。
Describe the solution you'd like
A clear and concise description of what you want to happen.
我希望这个扩展可以整合media pipe模型让它变得不用再下载。
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.
最后感谢你们的制作组，我还是第一次见到功能如此强大的扩展，结合onepose 和 candy模型，使用两个controlnet,居然不在会有手脚奇怪的问题。十分强大的扩展，感谢开源！

When I try to detect post from an image, it shows "Downloading MediaPipe Pose Model" and just sits there. How can I fix this manually? What do I need to download and where to place it?

Hi all,
It could be great to have the "detect pose from image" feature as we have in https://zhuyu1997.github.io/open-pose-editor/
It will allow us to "import" a pose from an image (as we can do on ControlNet) but we can edit the pose in 3D directly in the editor.

Original open pose has a body center point which allows you to rotate the entire body relative to the ground plane. Also I can't work around it, because I can't rotate the camera around it's z axis.

Is your feature request related to a problem? Please describe.
nope

Describe the solution you'd like
I used the random pose button a lot- and it was very convenient that it was not hidden in a drop down menu before. Please bring it back to it's former glory. :)

Describe alternatives you've considered
Blackmail, threats, manipulation, yelling in all caps, denial, resignation, acceptance

Additional context
The random pose button now requires multiple clicks to get to, and it is tedious to get to if you use it a lot. If the UI was less animated-ish and more responsive it would be less of an issue, but it would still be annoying :/

Thanks :)

When using the extension i mostly always start with a certain base pose. The upload and download file option is too complicated and slow in my opinion. Something like the presets in the openpose-editor would be great.

Could you implement that?

99f3e80
When I press X or select 'Move mode' in the menu, the active node changes to the center node, common to the entire model. It is not possible to switch to other nodes.

The editor is more and more complex.
If only provide webui bind by some buttons is not enough, and need to make many changes upstream.
I request to reduce some buttons and use the interaction of the editor self.

I just installed this extension and the viewport is just white and it doesn't seem like any of the buttons do anything. I haven't had any problems with other extensions. The viewport in PoseX works fine.

I installed with the "Install from URL" tab in Automatic1111.
Did I miss something?

Thanks!

Read the open pose json error downloaded by civitai

骨骼图画出来头总是向上仰

I got good results with inpaining mask manually, it would be helpful to have a "Send to Inpaint" and "Send to Inpaint upload" button

maybe hand.gbx. The file path seems to be incorrect. please help.

ERROR: Exception in ASGI application

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/starlette/responses.py", line 335, in call
stat_result = await anyio.to_thread.run_sync(os.stat, self.path)
File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
FileNotFoundError: [Errno 2] No such file or directory: '/content/repository/extensions/sd-webui-3d-open-pose-editor/models/hand.fbx'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.9/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/fastapi/applications.py", line 273, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/gzip.py", line 24, in call
await responder(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/gzip.py", line 44, in call
await self.app(scope, receive, self.send_with_gzip)
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.9/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/usr/local/lib/python3.9/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/routing.py", line 69, in app
await response(scope, receive, send)
File "/usr/local/lib/python3.9/dist-packages/starlette/responses.py", line 338, in call
raise RuntimeError(f"File at path {self.path} does not exist.")
RuntimeError: File at path /content/repository/extensions/sd-webui-3d-open-pose-editor/models/hand.fbx does not exist.