[Feature Request] TensorRT support about comfyui

[comfyanonymous]

Yes it's planned. There's a few more things I want to implement before. Unless something better comes out in the meantime I'm going to implement it.

[nistvan86]

SDXL models now have a TensorRT variant. https://huggingface.co/stabilityai/stable-diffusion-xl-1.0-tensorrt

[Erehr]

Aaaand new drivers just dropped promising 2X performance boost with TRT
https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/

[xueqing0622]

support!
https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

[phineas-pta]

advanced users can try my node: https://github.com/phineas-pta/comfy-trt-test

[Dunc4n1dah0]

I just tested Nvidia's A1111 TRT extension and results are 2x faster indeed (at least for 512x768 simple generation).

[78Alpha]

There was a pull for automatic that references the limits of TRT.

AUTOMATIC1111/stable-diffusion-webui-tensorrt#36

There is an upper limit of what it can do. As an example, if you have your batch size set to 8, you may not be able to generate dynamic images greater than 512x480 and the like.

I took note of some issues I've found during the year of TRT implementations flooding GitHub.

AUTOMATIC1111/stable-diffusion-webui-tensorrt#46 (comment)

Max dimensions I got from the calculated limit is Batch size 1, 858 x 858.

[78Alpha]

Be prepared to pull your hair out if you get to it. Getting it setup on one machine is a chore, but getting it to work on someone else's is worse. Many TensorRT implementations have fallen because of it. Volta-ML and SDA-node are the larger examples.

[CyberTimon]

I would love to see tensorrt support, because SD XL is quite slow.
Also, tensorrt only seems to support max 768x768px. Do you think it is somehow possible to pass the SDXL 1024 in?

[comfyanonymous]

Yeah I'm going to go with AITemplate instead of TRT unless they add a way to replace the weights at runtime.

[CyberTimon]

Yeah AiTemplate seems good too.

[al-swaiti]

yeah its there! any idea how make it work in comfyui

[78Alpha]

Still has the aforementioned issues with being dynamic (size, maximums, minimums). But, it's at least an official implementation for A1111. Hopefully that means it's working now, and on windows.

…

On Tue, Oct 17, 2023, 10:23 AM Erehr ***@***.***> wrote: Aaaand new drivers just dropped promising 2X performance boost with TRT https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/ — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7OOITKIRYIJOLHOCKSSLLX725H7AVCNFSM6AAAAAAVIY5CK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRWHA2TIMJWHA> . You are receiving this because you commented.Message ID: ***@***.***>

[BassJMagan]

Aaaand new drivers just dropped promising 2X performance boost with TRT https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/

and with that I'd love to see the fruits of this 2x speed boost as that could greatly improve my workflow on my 3090.

[SplitMilky]

Ok so how do i translate that support instructions for auto1111 to compfy. As I was using auto1111 but I was getting issues creating and using the optimized versions

[robinjhuang]

@BlueAccords Have you tried this?
https://github.com/comfyanonymous/ComfyUI_TensorRT