[unleaded]

Qwen3.6-35B-A3B-UD-Q4_K_M runs at about 11 tokens/second on my poor old 1060. Absolutely nuts how far we've come

[piyh]

I tried running any model on my 1070 and it instantly crashes my old tower, probably time to get off windows and run linux on it.

[SV_BubbleTime]

Understated how much of a boon for Linux that AI development has been.

There isn’t any benefit to running a windows machine.

[selectodude]

Au contraire, I run models on WSL and my desktop reliably wakes up from sleep. Best of both worlds.

[scotty79]

Watch out for VRAM/RAM. When programs on WSL tired to take more than there was available it was crashing my WSL hard and even corrupting files on the virtual drive attached to WSL in a very strange manner that made recent files just disappear. I had at leat two projects that had to be rebuilt from past conversation with AI because all their files were gone after few crashes.

I asked codex to write WSL config to prevent crashes. It put some limits on WSL and the situation stabilized, but I lost all trust for WSL anyways.

[fallinditch]

What are the config settings you needed?

TIA

[scotty79]

That's my wslconfig file located in my %USERPROFILE% dir:

    [wsl2]
    memory=40GB
    processors=8
    swap=8GB
    swapFile=C:\\wsl-swap.vhdx
    localhostForwarding=true
    guiApplications=false
    vmIdleTimeout=30000
    maxCrashDumpCount=3
    
    [experimental]
    autoMemoryReclaim=gradual

I have 64GB of RAM and 24GB of VRAM.

[MrDrMcCoy]

Genuine question: what are the good parts of Windows that keep you there? I've been such a happy gamer/coder/media editor on Linux for so long that I really don't understand how Microsoft isn't dead yet.

[selectodude]

My desktop is a game console and the games I play work on windows. And my computer reliably wakes from sleep.

[MrDrMcCoy]

My Linux laptop and desktop both handle gaming and waking from sleep just fine. I'd be more surprised if they didn't. You might like CachyOS or Bazzite if you try it.

[greenavocado]

Sounds like a hardware issue, though NVIDIA driver issues can't be ruled out, they're much rarer these days

[broodbucket]

Mind sharing your llama.cpp settings for that?

[unleaded]

  .\llama-server.exe -m ..\Qwen3.6-35B-A3B-UD-Q4_K_M.gguf -ngl 999 --n-cpu-moe 41 -c 262144 --port 8081 --flash-attn on --cache-type-k turbo4 --cache-type-v turbo3 --no-mmap --mlock --host 0.0.0.0 -t 8 -tb 8 -np 1

Using this llama.cpp fork https://github.com/TheTom/llama-cpp-turboquant and mostly copying from this video https://www.youtube.com/watch?v=8F_5pdcD3HY

Haven't had much time to test it other than asking a few questions & changing some HTML in cline so it might be thick as a brick for all I know, but still worth trying

[unleaded]

I just tested it with some risc-v code and it wrote down a "mov" instruction several times.. yeah something needs tuning maybe