I don't see a difference on GTX 1080

[MastaDan9]

I have gone through all the steps described in examples\directml\stable_diffusion.
I also installed the latest Nvidia drivers.
Launched marked --optimizer on the standard stable-diffusion-v1-5 model. After Optimizing the ONNX pipeline, I launched with the note "stable_diffusion.py --num_inference_steps 50 --num_images 1", it took me 24-26 seconds to generate, but without ONNX, I also get 24-26 seconds. Shouldn 't there be acceleration ?

[jstoecker]

Hi @MastaDan9. The NVIDIA improvements require RTX GPUs with tensor cores (outlined in this DirectX post and NVIDIA post). The GTX cards do not have tensor cores, unfortunately, so you should not expect to see the same performance improvement.

[Dampfinchen]

Hi @MastaDan9. The NVIDIA improvements require RTX GPUs with tensor cores (outlined in this DirectX post and NVIDIA post). The GTX cards do not have tensor cores, unfortunately, so you should not expect to see the same performance improvement.

Are RTX 20 series GPUs also benefitting from these improvements? This is pretty confusing as in the blog post Microsoft mentions RTX 30 and 40 series support, but Nvidia in their own blog post states all RTX GPUs are supported. RTX 20 series have tensor cores, so shouldn't they benefit from it as well?

[jstoecker]

Are RTX 20 series GPUs also benefitting from these improvements?

I just tested with an RTX 2080, and the 20 series cards are also benefitting from the 532.03 NVIDIA driver.

It's a holiday weekend in the US right now, but we'll update the post to reflect this when people are back next week. Thanks for pointing this out. @JacquesvanRhynMSFT FYI