Hacker News new | ask | show | jobs
by adrian_b 1037 days ago
I have compiled it with the command:

cmake .. -DGGML_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc

to use my NVIDIA GeForce RTX 2060 SUPER.

I have converted the model to use FP16.

With these choices, the time per iteration is between 8.5 s and 9 s and the total time for making an image is around 200 s.
1 comments
dragonwriter 1037 days ago
That seems a lot worse than a 2060 SUPER with PyTorch in A1111.

https://vladmandic.github.io/sd-extension-system-info/pages/... (search for 2060 SUPER)

link
adrian_b 1037 days ago
This is not surprising, because the GPU support in GGML is said to be preliminary and it is optimized for being run on CPUs.

Seeing the times reported by other people, it seems that using the GPU with GGML, instead of the CPU, still provides a speed improvement, but it is small.

Nevertheless, I have appreciated that after following exactly the instructions of this project everything was up and running after a few minutes and it could be tested.

Past attempts to install all the environment needed to run such models have required much more work.

link