olegchomp / tddepthanything

TouchDesigner implementation for Depth Anything and TensorRT monocular depth estimation.

Depth-Anything TensorRT in TouchDesigner

TouchDesigner implementation for Depth Anything and TensorRT monocular depth estimation.

Features

• One click install and run script
• In-TouchDesigner inference

Usage

Tested with TouchDesigner 2023.11340 & Python 3.11

Install:

1. Install Python 3.11
2. Install CUDA Toolkit 11.8
3. Download TDDepthAnything repository
4. Run accelerate.bat

Accelerate models:

1. Download Depth-Anything model
2. Copy model to checkpoints folder
3. Run accelerate.bat
4. Select model size (s - small, b - base, l - large)
5. Select width & height (default is 518x518)
6. Wait for acceleration to finish

TouchDesigner inference:

1. Add TDDepthAnything.tox to project
2. On Settings page change path to TDDepthAnything folder and click Re-init
3. On Depth Anything page select path to engine file (for ex. engines/depth_anything_vits14.engine) and click Load Engine

Acknowledgement

Based on the following projects:

• Depth-Anything - Unleashing the Power of Large-Scale Unlabeled Data.
• Depth-Anything TensorRT C++ - Leveraging the TensorRT API for efficient real-time inference.
• TopArray - Interaction between Python/PyTorch tensor operations and TouchDesigner TOPs.

tddepthanything's Issues

Can't import torch

Added 'C:\TDDepthAnything\venv\Lib\site-packages' to sys.path.
DAT compile error: /project1/TDDepthAnything/DepthAnythingExt
Traceback (most recent call last):
File "/project1/TDDepthAnything/DepthAnythingExt", line 28
r = previousimport(*args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\TDDepthAnything\venv\Lib\site-packages\torch_init_.py", line 141
raise err
OSError: [WinError 126] The specified module could not be found. Error loading "C:\TDDepthAnything\venv\Lib\site-packages\torch\lib\shm.dll" or one of its dependencies.

It seems there is an issue when importing pytorch

Has anyone solved that

Error on .tox Init

Hello!

Have problem with .tox component.
accelerate.bat has been running twice without errors.

DAT compile error: /project1/TDDepthAnything/DepthAnythingExt
Traceback (most recent call last):
  File "/project1/TDDepthAnything/DepthAnythingExt", line 1
    r = previousimport(*args, **kw)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Downloads/TDDepthAnything-main\.venv/Lib/site-packages\tensorrt\__init__.py", line 18
    from tensorrt_bindings import *
    r = previousimport(*args, **kw)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Downloads/TDDepthAnything-main\.venv/Lib/site-packages\tensorrt_bindings\__init__.py", line 141
    from .tensorrt import *
    r = previousimport(*args, **kw)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'tensorrt_bindings.tensorrt'

Error retrieving extension for /project1/TDDepthAnything: (Extension 1) td.tdError: Module compilation error. See /project1/TDDepthAnything/DepthAnythingExt for details.
Traceback (most recent call last):
  File "/project1/TDDepthAnything/script1_callbacks", line 18, in onCook
AttributeError: 'NoneType' object has no attribute 'run'`

Pulse btns gives the same error.

This plugin so amazing, but is there a way to fix the flickering problem when use the webcam stream？

Issues with instructions running accelerate.bat first time

Just something to note when running accelerate.bat for the first time on python 3.11.8:

error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pycuda
Failed to build pycuda
ERROR: Could not build wheels for pycuda, which is required to install pyproject.toml-based projects
Found existing installation: nvidia-cudnn-cu11 8.9.4.25
Uninstalling nvidia-cudnn-cu11-8.9.4.25:
Successfully uninstalled nvidia-cudnn-cu11-8.9.4.25

Also getting an issue after including the models in checkpoints about :

Preparing for model acceleration...
Traceback (most recent call last):
File "D:\Experimental\TDDepthAnything-main\TDDepthAnything-main\accelerate_model.py", line 5, in
from polygraphy.backend.trt import (
ModuleNotFoundError: No module named 'polygraphy'
Press any key to continue . . .

Engine not created. Ensure engine was created properly error

accelerate.bat - not working, but..

Hi,
just wanted to let you know that the accelerate.bat throws an error, as it can not find python.
However, it was suggested online to try "py" instead of "python" and that did work for me.
Maybe pother users have the same issue...
All the best
Stefan

Can I use webcam for real-time depth detection? I see there are checkpoints models, is this based on SD?

TopArray license not included in code

Hi I noticed that the license specified in TopArray was not included the code. Would you be able include it in the extension where the TopArray classes are used? Or if you loaded TopArray.py into the network and imported the module into your extension the license would be include without it being in your extension.

Ah I didn't realize you linked the the library on the readme. Thanks!