Replies: 5 comments 6 replies
-
|
This is tensorrt and it's not very useful for local SD UIs in it's current state. If you actually try what they published you'll see why. |
Beta Was this translation helpful? Give feedback.
-
|
I tried it with Auto but got a bunch of warnings/errors. In practice right now it doesn't look great but I also don't know much about this whole stuff so I thought it could be helpful to some of you guys. |
Beta Was this translation helpful? Give feedback.
-
|
With my 3090 and a dynamic 512-768 tensorRT with SD 1.5, i went from 7 iterations per sec in automatic to 20. So now i'm hella exited to update drivers and see if i'd get 2x of that again. So 40 iterations per sec xD |
Beta Was this translation helpful? Give feedback.
-
|
I don't know what's not useful there (except that the development is fresh and there are not many contributors), but on 4090 we really got x2 speed. 512x512 was 22-24 it/s up to 49 it/s. It's freaking magic. |
Beta Was this translation helpful? Give feedback.
-
|
What comfy is talking about is that it doesn't support controlnet, GLiGEN, or any of the other fun and fancy stuff, LoRAs need to be baked into the "program" which means if you chain them you begin accumulating a multiplicative number of variants of the same model with a huge chain of LoRA weights depending on what you selected that run, pre-compilation of that is required every time, etc. I started off in the world of stable diffusion with my 7900XTX running Nod.ai Shark which uses a similar methodology except with the further restriction of not being able to generate arbitrary output size models and with huge (3GB each) generated models and only one LoRA. I could do SD 2.1 generation @ 512x512 at nearly 30 it/s before they broke it and around 25it/s in SD 1.5. After trying tons of models and loras out I realized I had 200GB of disk space being eaten by the precompiled models (and the original still had to be kept or they wouldn't work). After I learned about all the other things that could be done I installed Comfy, went through the painful process of making DirectML work, and had so much fun with QRCode Monster and all the latent composition type stuff that I was happily running things at 6s / it for the scaled up end stages and 1it/s earlier on, taking 3 minutes per image instead of 2s and never really went back to shark. That's what needs to be unbroken before that type of system will be generally useful. Edit: After several months of that and AMD botching the Windows HIP release by not having MIOpen ready, I bought an RTX4090 and now I don't have a bunch of compatibility issues with add-on modules either. Since the 7900XTX is useful for other things (it's still quite a bit faster for FP16 since the only mode of that which sees a speed increase on ada is fp16 tensor with fp16 accumulator whereas it's always double speed on RDNA3, for example) and my power supply can handle both at full tilt I'm keeping it around for now. |
Beta Was this translation helpful? Give feedback.
-
|
I'd expect TransformerEngine (the FP8 autocast stuff) to do about the same (at least on Ada) but damned if I can get it to build on Windows :P |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for sharing ! I really love comfyui. For now it seems that nvidia foooocus(ed) (lol, yeah pun intended) on A1111 for this extension. EDIT : got a bunch of errors at start. I don't understand the part that need some "export default engine" part. |
Beta Was this translation helpful? Give feedback.
-
|
Is it possible that improvements are always and only made for those who already have graphics cards that cost 600 euros or more installed on their computers? And for those who can't afford to have a powerful computer and only use the CPU to generate images, are improvements never made? |
Beta Was this translation helpful? Give feedback.
-
|
runpod.io is your answer :) |
Beta Was this translation helpful? Give feedback.
-
|
Well, I just checked A1111 on my 4060TI 16GB, installed https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT without single problem or error... and (all same settings, just changed sd_unet to TRT, made few tries of each), results are: Results are... OMG. We need this in Comfy :-) |
Beta Was this translation helpful? Give feedback.
-
|
👍🏻 thank you very much
Il Ven 27 Ott 2023, 06:45 ManuiaGeek ***@***.***> ha scritto:
… runpod.io is your answer :)
Get your RTX 4090 24 GB Starting at $0.74 / hr !
—
Reply to this email directly, view it on GitHub
<#1775 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A57JPAI6HLLPAP7BTNIEOWLYBM36JAVCNFSM6AAAAAA6EFHHYCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TGOJZGQ2TS>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
NVIDIA just released the 545.84 drivers and they show a 2x speed improvement on image inference on this page:
https://www.nvidia.com/en-us/geforce/news/game-ready-driver-dlss-3-naraka-vermintide-rtx-vsr/
Reading a bit about it it seems to require this to work https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT, the readme only mentions AUTOMATIC1111 so I guess it will need work for it to wok in ComfyUI.
More infos here: https://nvidia.custhelp.com/app/answers/detail/a_id/5487
I did a few tests with the new drivers only and there is no improvement for me between 537.34 and 545.84 with a 3060 12GB.
Beta Was this translation helpful? Give feedback.
All reactions