[vhold]

That's what this article is about, they're doing the integration work.

An integration with a very popular tool is already available for use. I've personally tested it and did get a 2x speed up. I didn't have to sign-up for anything.

https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

[brucethemoose2]

Ugh... I know everyone uses automatic, but I don't really like automatic. Is janky as heck, especially with SDXL, especially if you want to dig into the code and make some manual hacks.

Still, you are right, a native a1111 extensions is quite impressive and exactly the kind of thing Nvidia should pursue.

[diggan]

diffusers also has example usage for TensorRT in the Stable Diffusion context: https://github.com/huggingface/diffusers/blob/85dccab7fd280c...

Seems those particular examples were submitted by NVIDIA employee too: https://github.com/huggingface/diffusers/commit/9d44e2fb6600...

Shouldn't be too hard to adopt to other similar pipelines if you've used diffusers before.

[brucethemoose2]

That's a good example of a "hit and run" demo.

None of the community Diffusers pipelines I know of have successfully integrated this. VoltaML made a valiant effort and gave up in favor of Facebooks AITemplate (for now).

[diggan]

Worked for me and was easy to integrate, but I use diffusers directly. Not sure what the others are doing, but if you can manage to implement your own pipeline with diffusers, you should be able to integrate the TensorRT pipeline with relative ease.

https://github.com/VoltaML/voltaML seems to have successfully integrated it as far as can tell, it says TensorRT is one of the compatible backends.

[brucethemoose2]

VoltaML abandoned it (but may revisit it) and replaced it with AIT.

The problem with a vanilla diffusers pipe is that you miss out on all the cool community augmentations that really make sd shine. Crazy img2img stuff aside, some random examples include:

- Fooocus's implementation preserves 'momentum' when switching from the sdxl base model to the refiner.

- freeu augmentation

- tomesd

- fooocus image style presets, with known styles from the dataset strait from SAI and others.

- auto prompt expansion

- performance hacks not in vanilla diffusers

[qiine]

What do you use instead? confyUI ?

[brucethemoose2]

Fooocus MRE for SDXL. The quality is just so much better than a1111 or vlad's fork. And its stable, with a good UI:

https://github.com/MoonRide303/Fooocus-MRE/commits/moonride-...

I use VoltaML for SD 1.5: https://github.com/VoltaML/voltaML-fast-stable-diffusion/tre...

Next on my todo list is trying ComfyUI with AIT: https://github.com/FizzleDorf/ComfyUI-AIT

[qiine]

so many projects... its overwhelming

[brucethemoose2]

FYI I just learned that Fooocus-MRE has been depreciated, but many of the same options are now in the debug menu upstream.

...Yes, it is impossible to keep up, lol

[nickthegreek]

ComfyUI or fooocus handle SDXL way better than A1111 with 10gb VRAM or under. While I really like A1111, it just isnt usable with XL for me.

[brucethemoose2]

I run them on a 3090, and fooocus is still way better.