[GordonS]

Looks great! Does it work on Windows please?

[buzzier]

For Windows:

1. Binary build https://github.com/jaykrell/llama.cpp/releases/tag/1

2. Quantized model (7B/13B/30B) https://mega.nz/folder/UjAUES6Z#bGhKkyiZX3eRrn9HcxVVfA

3. main.exe -m ggml-model-q4_0.bin -t 8 -n 128

[placebo]

Thanks. Initial test:

main.exe -m ggml-model-q4_0.bin -t 8 -n 128 -p "The Drake equation is nonsense because"

The Drake equation is nonsense because it takes parameters that can only be known AFTER the conclusion is reached. It would be like saying "I'm going to prove a theorem by starting from the conclusion, then making up the proof. The Drake equation uses the existence of extraterrestrial intelligence as the conclusion and then making up the parameters. It is nonsense.

[GordonS]

Nice, main.exe seems to work just fine with the 7B quantized model - generates a token every 400ms on an AMD Ryzen 5 2600!

But, quantize.exe doesn't seem to work - any valid command (such as below) pauses for a split second, then returns with no output?

$ quantize.exe ggml-model-f16.bin ggml-model-q4_0.bin 2

[GordonS]

In case this helps anyone else: I built it myself on Windows with CMake, and then everything just works.

[starik36]

Do you mind sharing the binaries?

[GordonS]

Sure! https://filetransfer.io/data-package/8hxKAiaH#link

I wasn't sure where to upload them, and that link is only good for 50 downloads. Can put them somewhere else if you know a better location that doesn't require signup.

[starik36]

Thank you.

llama.exe is basically main.exe?

I actually learned how to compile this code via CMake/VS2019. It's sure a whole lot more complicated then it was 25 years ago when I was writing C.

[tough]

I actually am installing in windows via WSL/Ubuntu fwiw

[bsenftner]

My attempt does not work, and now I'm trying to figure out where the 35+ GB of data and files that were added to my hard drive are located so I can clean it all off.

[tough]

I got it to work with WSL/Ubuntu in case you want to try it that way.

[dragonwriter]

If it makes common unix-ish assumptions like “Python 3 executables have a ‘3’ appended to their name”, which other comments here seem to suggest it does, it won’t, even if you have the required version of python installed.

[GordonS]

So, I actually got it working on Windows, pretty easily!

The provided `main.exe` binary worked as-is, but `quantize.exe` did not - I built myself with CMake, and `quantize.exe` started working too.

[volaski]

Curious too. Let me know if you try it out. Technically I think it should work.

[starik36]

I tried it, doesn't work. Trying the sibling post from @buzzier.