[dotancohen]

  > For 16GB laptops, Qwen 3.5 9B is the undisputed champ.

You seem like the guy to ask. For a laptop with 12GB VRAM (RTX 5070) and 32 GB system RAM, what is a good multilingual (English, Hebrew, Greek) model for conversing with personal notes in Org mode format? I don't care how long updating the model or rag takes, and even inference can be reasonably slow, but the results of the query as they relate to my personal notes are important. I don't care about general knowledge, for those questions I can use e.g. ChatGPT.

Thanks

[akmarinov]

Joins us over on Reddit at r/LocalLlaMA to get 10 different opinions on that

[dotancohen]

I read there regularly. I find little value there between the memes. I was hoping to ask a knowledgeable person here.

[alfiedotwtf]

/r/localllama for a while now seems to prefer Gemma 4 E4B for creative writing (especially the uncensored GGUFs).

[plagiarist]

Do they prefer E4B over the larger models or is it a matter of what fits their machine? I assume 4B isn't large enough to get interesting writing but I don't know anything about it.

[KludgeShySir]

Gemma4-31b seems to be very highly regarded for creative writing, especially its finetunes. As for comparison to E4B, I can't say.

Creative writing is not my focus, so this is only secondhand information. r/LocalLlama tends to focus more on the technical side; if you want more creative side check out r/SillyTavern as well (but type of info does bleed over between them).

[nl]

Qwen 3.5 35B A3

Qwen models are always good. The 35B A3 model is a MoE model which means it has higher performance in RAM constrained environments compared to the 27B dense model (which is better at coding).

I don't have experience to rate it's Hebrew or Greek performance but apparently it's not bad.

[sourcecodeplz]

Any Gemma 4 model, they are great at translations, multilingual

[silversmith]

For the biggest languages, Spanish, French, maybe.

For smaller ones like my native Latvian, the output could be confused for good translation from across the room, the words do look like Latvian words. But the quality is Google translate circa 20 years ago, tops.

It could probably do a decent enough translation to English, if all you need is to get the gist of text. But for smaller European language outputs, nothing comes close to Gemini.

[dotancohen]

While Gemini 4 seems fine, Gemma 4 does not do Hebrew well. I've replaced it with Aya Expanse and am getting much better results, but there is still much improvement to be had.

I'm not doing translations, rather querying Hebrew text with a Hebrew prompt.

[emmelaich]

You may like https://www.llmfit.org/

(not recommendation, I've not used it .. yet)

[hypfer]

Just tried it and honestly it's a terrible experience lacking any sort of intent or reason.

Which is unsurprising in the AI space.

You get a wall of text showing you various random fine-tuned models by random people, and that is basically it.

Actual sane default requirements like "just give me the normal AI labs", "please filter for dense only" and "I want this exact context size at this quant" are not part of the tool, apparently. Neither is "compare these quants for me for the same model".

Or maybe it's just hidden enough that I did not find them before I've stopped caring.

Conway's law is at it again.

____

Edit:

I have since then had qwen3.6 ponder the codebase and think about my complaints.

Seems to require a major data model overhaul to actually fix those, so they're legit. Which I didn't doubt, but nice to have some extra fabricated confirmation after it initially refused and said "nooooo the readme says otherwise nooo hypfer is just a hater noo"

___

Edit 2:

It gets worse the longer I stare at it. This could've been a web calculator.

[hypfer]

Done:

https://github.com/Hypfer/will-it-fit-llama-cpp

https://hypfer.github.io/will-it-fit-llama-cpp/

[hparadiz]

We need benchmarks by engine, cli switch sets, and device with filters by cpu, gpu, and type. And if someone could please aggregate that in a way where people can upload results and just automatically see the best of any model for their device that would be a killer app.

[alfiedotwtf]

I've wanted to vibe code a tuning app, that pumps data through your CPU-GPU-RAM to try and determine the best parameters for each model, but I think it's just too much work compared to manually running by hand a one-liner and changing things here and there.

[dofm]

I have found these things to be fully exasperating, to be honest, even though I am seeking information about a pretty "known" machine — a 64GB M1 Max MBP.

(Honestly I think Apple's "AI push" could do worse than just focus on a curated model library, a couple of Apple-standard Gemini distillations, an OS-level model manager and some sort of tweak of their containers system to do what Docker's sbx does. They could demystify a lot of this shit.)

[tacomagick]

Gemma 4 26A4B