It's okay I have to say. I just ran out of memory on my 4090, so I had to retry on an A100. Here's an extract: https://pastebin.com/pzLfCFWt
I think something might be off with the example. Can't wait for this stuff to work on llama.cpp. Going to try it with mistral & stable lm now, thankfully tomorrow is a holiday in Germany :)
I think something might be off with the example. Can't wait for this stuff to work on llama.cpp. Going to try it with mistral & stable lm now, thankfully tomorrow is a holiday in Germany :)