Question: about FP8 support for Ada and Hopper.. about voltaml-fast-stable-diffusion

[oscarbg]

Edit: I asked about tensorrt fp8 support here:
NVIDIA/TensorRT#2730

[Stax124]

Hey @oscarbg, tbh I just postponed the TensorRT pipeline indefinitely after spending hundreds of hours debugging it on my sytem, current development and focus is on AITemplates - so - even if I manage to somehow get TRT up and running again, there is almost zero chance that we can just upgrade all the dependencies (which were the issues that I was having).

If some major help comes or I get a GPU with more VRAM then I might reignite the work there - but it is highly unlikely.

Anyways, if you want to try AIT, come and join our discord, steps on how to run are in the announcement channel (or I will probably update the docs tomorrow).

(and yes, if F8 becomes possible than you are looking at about 2x speedup - with a lot of quality loss i guess)

Have a nice rest of your day.

[oscarbg]

Thanks @Stax124 for all the info..