brokensource / depthflow Goto Github PK

🌊 Image to → 2.5D Parallax Effect Video. High quality, user first

License: GNU Affero General Public License v3.0

GLSL 41.85% Python 58.15%

depth-prediction image-to-video monocular-depth monocular-depth-estimation parallax image-parallax parallax-effect depth-map depthy computer-vision depth-maps shadertoy leiapix

depthflow's Issues

(BUG) Segmentation Fault on Apple Silicon when Compiling Shaders

Hi I am running Depthflow on Mac Silicon and currently whenever I run depthflow I got

zsh: segmentation fault depthflow any ideas?

how to open after the installation?

how do i open it after installing it in my system?
shaderflow is also needed or not?

Can't select input image through comandline or file selection dialog

Thanks for this nice project.

Sadly, I always get this mountain rendered but did not manage to set my own input image. If using drag&drop to the preview, I can make it work, but then I get no rendered output file.

I tried using -i or --image, but this gives me error that I should check help. The help page does not tell me how to set the input image.

$ broken depthflow -i DoA_DiamondValley.jpg
...
Usage: main [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...                                                                                     
                                                                                                                                                      
No help provided
...

I then tried with parallax:

$ broken depthflow parallax -i DoA_DiamondValley.jpg
Exception in thread Thread-2 (__parallax__):
Traceback (most recent call last):
  File "/usr/lib64/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/home/volker/dummy/BrokenSource/Projects/DepthFlow/DepthFlow/DepthFlow.py", line 40, in __parallax__
    self.__load_depth__ = BrokenUtils.load_image(depth or self.mde(image, cache=cache))
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/volker/dummy/BrokenSource/Projects/DepthFlow/DepthFlow/Modules/DepthFlowMDE.py", line 46, in __call__
    image_hash = hashlib.md5(image.tobytes()).hexdigest()
                             ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'tobytes'

Is there something I missed or doing wrong?

I also do not get any file selection dialog opening up like the github page is saying? It just starts with the mountains.

I'm on OpenSUSE LEAP 15.5 x86_64 with most recent updates.

DepthFlow cannot be deployed and run

I’m trying to use the DepthFlow project, but it depends on ShaderFlow. After copying ShaderFlow into the projects, I found it depends on Brokens. However, I couldn’t find Brokens under https://github.com/BrokenSource. The entire project cannot run properly. Also, I would like to ask if there is a complete plan for using DepthFlow?

(BUG) Quality went downhill in latest version

Hello,

Today I downloaded the latest version and tried to run the code, but the result is significantly worse than with the code from around 3 weeks ago.

Here is the command that I run with the new code: depthflow input --image ./image.png main -w 1080 -h 1920 -s 8 -f 30.0 -q 100 -r -o ./output.avi --format avi -f 30
Here is the command that I run with the old code: broken depthflow parallax --image image.png main --format avi -q 100 -s 8 -w 1080 -h 1920 -f 30.0 -r -o ./output.avi

The new and low-quality result (see the distortion) can be seen here:
https://streamable.com/ri72fk

The old result can be seen here. While it was not perfect, there was significantly less "stretching".
https://streamable.com/y05h7u

Here is the image, in case you want to test (it was too big to upload here):
https://we.tl/t-yXmBj7zOjZ

(ignore the image quality, it is something I quickly generated with stable diffusion to test the new version)

(REQUEST) A way of executing this without launchin UI

Is it planned to add a CLI only mode to this? This would be nice to have.

(BUG) error after update to broken-source 0.3.1

~ >pip list
Package                Version
---------------------- --------------
broken-source          0.3.1
depthflow              2024.1.15

~ >depthflow input -i C:\Users\steve02081504\Downloads\116018494_p0.png main

---------------------------------------------------------------------------------------
198 _run_module_as_main <frozen runpy>
>

88 _run_code <frozen runpy>
>

4 <module> E:\Python\Scripts\depthflow.exe\__main__.py
> from DepthFlow.__main__ import main

4 <module> E:\Python\Lib\site-packages\DepthFlow\__main__.py
> from DepthFlow.DepthFlow import DepthFlowScene

21 <module> E:\Python\Lib\site-packages\DepthFlow\DepthFlow.py
  from ShaderFlow.Optional.Monocular import DepthEstimator
  from ShaderFlow.Scene import ShaderScene
  from ShaderFlow.Texture import ShaderTexture
  from ShaderFlow.Variable import ShaderVariable
  from typer import Option

  from Broken import image_hash
  from Broken.Externals.Upscaler import BrokenUpscaler
  from Broken.Externals.Upscaler.ncnn import BrokenRealEsrgan
  from Broken.Loaders import LoaderImage
> from DepthFlow import DEPTHFLOW


  class DepthFlowState(BaseModel):

      height: float = Field(default=0.35)
      """Peak value of the Depth Map, in the range [0, 1]. The camera is 1 distance away from depth=0
      at the z=1 plane, so this also controls the intensity of the effect"""

      focus: float = Field(default=0.0)
      """Focal depth of offsets, in the range [0, 1]. A value of 0 makes the background (depth=0)

ImportError:
cannot import name 'DEPTHFLOW' from 'DepthFlow' (E:\Python\Lib\site-packages\DepthFlow\__init__.py)
~ >vat E:\Python\Lib\site-packages\DepthFlow\__init__.py
cat E:\Python\Lib\site-packages\DepthFlow\__init__.py [enter/↑/↓/ctrl+c]
# Reserving pypi name
~ >

Can you make a website?

Recently it is working fine.

same image again

2024-04-15_16-27-34.DepthFlow.mp4

Lookin fine from the last one but i think i still missing some settings.
i used this command
depthflow input --image (url | path) main --render -s 2

so i want to ask you that is it possible to make a opensource website just like "Depthy". That will be so much easier for everyone.

Render video from one cli command directly from image and depth map

Hello,
really nice little tool, would it be possible to directly render the video using image and depth map directly from cli?

(REQUEST) support conda or pip install

Hello,
Very nice tool, would it be possible to support conda/pip installation?

i can't find the perfect settings

hi can you please help me to find the perfect settings for depthflow video. Please use my image.

Its hard for me to find the perfect settings. A little help can make my work easier

Can you add a version that runs on Linux servers?

I have a server but no interface, I can only use bash, all the OSError when downloading the model, it gives me a headache to find a solution, please can you release a version that can run directly in the terminal as soon as possible.

(BUG) 'NoneType' object has no attribute 'get'

 >depthflow input -i C:\Users\steve02081504\Downloads\116018494_p0.png main
│DepthFlow ├┤ 732ms├┤INFO   │ ▸ ( 1) DepthFlowScene     │ ▸ Module added to the Scene
│DepthFlow ├┤ 736ms├┤INFO   │ ▸ ( 1) DepthFlowScene     │ ▸ Creating glfw Window
│DepthFlow ├┤ 834ms├┤INFO   │ ▸ ( 1) DepthFlowScene     │ ▸ Adding default base Scene
modules
│DepthFlow ├┤ 835ms├┤INFO   │ ▸ ( 2) ShaderFrametimer   │ ▸ Module added to the Scene
│DepthFlow ├┤ 836ms├┤INFO   │ ▸ ( 3) ShaderKeyboard     │ ▸ Module added to the Scene
│DepthFlow ├┤ 837ms├┤INFO   │ ▸ ( 4) ShaderCamera       │ ▸ Module added to the Scene
│DepthFlow ├┤ 838ms├┤INFO   │ ▸ ( 5) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 839ms├┤INFO   │ ▸ ( 6) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 840ms├┤INFO   │ ▸ ( 7) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 841ms├┤INFO   │ ▸ ( 8) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 842ms├┤INFO   │ ▸ ( 9) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 842ms├┤INFO   │ ▸ (10) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 843ms├┤INFO   │ ▸ (11) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 844ms├┤INFO   │ ▸ (12) ShaderDynamics     │ ▸ Module added to the Scene
│DepthFlow ├┤ 845ms├┤INFO   │ ▸ ( 1) DepthFlowScene     │ ▸ Creating SSAA
Implementation
│DepthFlow ├┤ 846ms├┤INFO   │ ▸ (13) ShaderObject       │ ▸ Module added to the Scene
│DepthFlow ├┤ 847ms├┤INFO   │ ▸ (14) ShaderTexture      │ ▸ Module added to the Scene
│DepthFlow ├┤ 851ms├┤INFO   │ ▸ (15) ShaderObject       │ ▸ Module added to the Scene
│DepthFlow ├┤ 852ms├┤INFO   │ ▸ (16) ShaderTexture      │ ▸ Module added to the Scene
│DepthFlow ├┤ 858ms├┤INFO   │ ▸ (17) ShaderTexture      │ ▸ Module added to the Scene
│DepthFlow ├┤ 860ms├┤INFO   │ ▸ (18) ShaderTexture      │ ▸ Module added to the Scene
│DepthFlow ├┤1248ms├┤SUCCESS│ ▸ DepthMap already cached on
(C:\Users\steve02081504\AppData\Local\BrokenSource\DepthFlow\Cache\b18ab620-3e48-6a72-4
a3f-b0b8d309b216.depth.png)
│DepthFlow ├┤1286ms├┤SUCCESS│ ▸ Already upscaled to target size (0, 0):
<PIL.Image.Image image mode=L size=518x518 at 0x1FC6D6B18B0>
│DepthFlow ├┤1287ms├┤SUCCESS│ ▸ Already upscaled to target size (0, 0):
<PIL.PngImagePlugin.PngImageFile image mode=RGB size=2048x2048 at 0x1FC5157F260>

---------------------------------------------------------------------------------------
198 _run_module_as_main <frozen runpy>
>

88 _run_code <frozen runpy>
>

7 <module> E:\Python\Scripts\depthflow.exe\__main__.py
> sys.exit(main())

10 main E:\Python\Lib\site-packages\DepthFlow\__main__.py
> depthflow.cli(sys.argv[1:])

574 cli E:\Python\Lib\site-packages\ShaderFlow\Scene.py
> self.broken_typer(args)

111 __call__ E:\Python\Lib\site-packages\Broken\Core\BrokenTyper.py
> raise error

105 __call__ E:\Python\Lib\site-packages\Broken\Core\BrokenTyper.py
> self.app(args)

326 __call__ E:\Python\Lib\site-packages\typer\main.py
> raise e

309 __call__ E:\Python\Lib\site-packages\typer\main.py
> return get_command(self)(*args, **kwargs)

1157 __call__ E:\Python\Lib\site-packages\click\core.py
> return self.main(*args, **kwargs)

723 main E:\Python\Lib\site-packages\typer\core.py
> return _main(

193 _main E:\Python\Lib\site-packages\typer\core.py
> rv = self.invoke(ctx)

1719 invoke E:\Python\Lib\site-packages\click\core.py
> rv.append(sub_ctx.command.invoke(sub_ctx))

1434 invoke E:\Python\Lib\site-packages\click\core.py
> return ctx.invoke(self.callback, **ctx.params)

783 invoke E:\Python\Lib\site-packages\click\core.py
> return __callback(*args, **kwargs)

692 wrapper E:\Python\Lib\site-packages\typer\main.py
> return callback(**use_params)

722 main E:\Python\Lib\site-packages\ShaderFlow\Scene.py
> video_resolution = self.resize(width=width, height=height, scale=scale, aspect_ratio=aspect)

477 resize E:\Python\Lib\site-packages\ShaderFlow\Scene.py
> self.window.size = self.resolution

145 size E:\Python\Lib\site-packages\moderngl_window\context\glfw\window.py
> glfw.set_window_size(self._window, value[0], value[1])

1382 set_window_size E:\Python\Lib\site-packages\glfw\__init__.py
> _glfw.glfwSetWindowSize(window, width, height)

689 errcheck E:\Python\Lib\site-packages\glfw\__init__.py
> _reraise(exc[1], exc[2])

70 _reraise E:\Python\Lib\site-packages\glfw\__init__.py
> raise exception.with_traceback(traceback)

668 callback_wrapper E:\Python\Lib\site-packages\glfw\__init__.py
> return func(*args, **kwargs)

374 glfw_window_resize_callback ...site-packages\moderngl_window\context\glfw\window.py> super().resize(self._buffer_width, self._buffer_height)

756 resize E:\Python\Lib\site-packages\moderngl_window\context\base\window.py
> self._resize_func(width, height)

942 __window_resize__ E:\Python\Lib\site-packages\ShaderFlow\Scene.py
> self.relay(Message.Shader.Render)

42 relay E:\Python\Lib\site-packages\ShaderFlow\Module.py
> module.handle(message)

379 handle E:\Python\Lib\site-packages\ShaderFlow\Shader.py
> self.render()

366 render E:\Python\Lib\site-packages\ShaderFlow\Shader.py
> self.use_pipeline(self._full_pipeline())

356 use_pipeline E:\Python\Lib\site-packages\ShaderFlow\Shader.py
> self.set_uniform(variable.name, variable.value)

276 set_uniform E:\Python\Lib\site-packages\ShaderFlow\Shader.py

  @fragment.setter
  def fragment(self, value: Union[Path, str]):
      self._watchshader(value)
      self._fragment = value

  # # Uniforms

  def set_uniform(self, name: str, value: Any=None) -> None:
      # Note: Denum safety, called hundreds of times: No noticeable performance impact (?)
>     if (value is not None) and (uniform := self.program.get(name, None)):
          uniform.value = denum(value)

  def get_uniform(self, name: str) -> Any | None:
      return self.program.get(name, None)

  # # Rendering

  def _full_pipeline(self) -> Iterable[ShaderVariable]:
      for module in self.scene.modules:
          yield from module.pipeline()

AttributeError:
'NoneType' object has no attribute 'get'

(BUG) Worse quality when using cached depthmap

Noticed a worse quality when using cached depthmap. Example video below, the left was generated when there was no depthmap yet, the right video is using the came cmd, but depthmap was loaded from cache. You can see jagged edges (ears, hands) compared to the first run.

My cmd:
depthflow input -i https://img.bricklink.com/ItemImage/MN/0/ani004.original.png -b main --ssaa 2.0 -q 100 --render --output ani004.mp4 --format mp4 --open

output.mp4

"Grab the latest DepthFlow Release for your platform, run it" - no release available.

Hello, I would like to try this project but as the title says, under the section "Grab the latest DepthFlow Release for your platform, run it" there is no release(s) available.

As well, are there any examples available for what exactly this does? Going by what's provided it sounds like it should take an image, generate a depth map and then animate it - but how exactly?

Awesome project

This project is absolutely awesome. The inference works so fast. I am thinking to integrate it to stable diffusion ui. Currently there is a similar plugin for the sd ui called https://github.com/thygate/stable-diffusion-webui-depthmap-script

But that one is painfully slow, first create a depthmap, then a mesh and then creates a video from the mesh. The mesh generation can take like 2-3 minutes and only works with CPU. (once generated the mesh you have to wait another time for video generation) And the result is almost the same as your project. With the clear difference that the results of your project are much better in quality, speed and also it uses CUDA.

So, congratulations for your project!

David Martin Rius

(BUG) GLIBC not found when executing new PyApp release on linux

I'm trying the newly packaged release and having problems with GLIBC not found. Currently I am unable to upgrade the operating system:

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"

This happens with all linux executables: CUDA, CPU and Rocm.
Is there a better way to run this in headless mode from command line and that does not require GLIBC? Other installation ways are quite convoluted and depend on RYE or the BrokenSource itself.

The exact errors are:

./depthflow-cpu-linux-amd64-0.3.1.bin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./depthflow-cpu-linux-amd64-0.3.1.bin)
./depthflow-cpu-linux-amd64-0.3.1.bin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./depthflow-cpu-linux-amd64-0.3.1.bin)
./depthflow-cpu-linux-amd64-0.3.1.bin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./depthflow-cpu-linux-amd64-0.3.1.bin)

And the command executed was:
./depthflow-rocm-linux-amd64-0.3.1.bin input -i ./blabla.png main -q 90 -s 2 -o ./video.mp4

Installation is so painful

My man. This seems like a fantastic project and I've been trying to get it running for the last 2 days.

It's been a constant roadblock of errors everywhere. Admittedly, not all of it comes down to your package (I'm on a headless server without admin rights), but some of the installation process is quite ridiculous.

Why do I need to install broken-source & every package you've created? Why are there sound libraries in a project which doesn't need sound? Why does installation take over 30mins?

I feel this could be as simple as "git clone depthflow" and then "depthflow --image X", with only the dependencies that are required for this project.

Or, stick your model on huggingface spaces/replicate. I know you may have a lot on your plate but the exposure you'd get by doing this might encourage some open source contributors to help out.

This is the first model of its kind on github (that I could find) -- make getting started with it easy and you'll be the main repo for this stuff

Changing the video codec

Hello!

I have one quick question for you, is it possible to change the video codec used when saving the effect to a video? Currently, it appears it is using the h264 codec, and I would like to use the png codec.

If there is no such feature, could you please point me to where in the code I can make this change?

Thank you!

Issue when trying to run Broken Depthflow

Hi,

First, thanks so much for making this, looks like it's got a lot of potential however I'm unable to get it to run.

I'm getting an AttributeError and Binary doesnt exist or was not foubnd on PATh as shown in the following when trying to run broken depthflow. I've tried both on my M1 Mac and Linux. x86.

Thanks!

ERROR │ ▸ Binary doesn't exist or was not found on PATH (/Users/deepdey/Library/Caches/pypoetry/virtualenvs/depthflow-E2DdtqXj-py3.10/bin/main)

.
.
.

Users/deepdey/Media_Local/depthflow2/BrokenSource/Broken/main.py
)

C++ projects
if self.is_cpp:
log.error("C++ projects are not supported yet")

Avoid reinstalling on future runs
reinstall = False

Detect bad return status, reinstall virtualenv and retry once
if (status.returncode != 0) and (not reinstall):
log.warning(f"Detected bad Return Status ({status.returncode}) for the Project ({self.name}) at ({self.path})")
if self.is_python:
log.warning(f"• Python Virtual Environment: ({venv})")
log.warning(f"• Command: {tuple(status.args)}")

  Prompt user for action
  import rich.prompt
  answer = rich.prompt.Prompt.ask(
      f"• Action: Run poetry (i)nstall, poetry (l)ock, (r)einstall venv, (e)xit or nothing (enter), then retry",
      choices=["r", "e", "p", "l", ""],

AttributeError:
'NoneType' object has no attribute 'returncode'

brokensource / depthflow Goto Github PK

depthflow's Issues

Recommend Projects

Recommend Topics

Recommend Org