[nsbk]

Bring it on! Hoping that they release smaller sizes of Qwen3.8. I use the 35B MoE and 27B dense models locally and most of the time I don’t need to reach out to Claude. Extremely useful specially when requests include sensitive and/or personal data

[mft_]

I think everyone is hoping this!

It would be great if they'd release an MoE model somewhere between the 35B size of 3.6 and the 122B version of 3.5 - it could be a great balance of speed and ability for people with reasonably powerful but not insane home computers.

[embedding-shape]

> the 122B version of 3.5

Yeah, this is what I'm holding out for, the NVFP4 variant of 3.5 122B is blazing fast with reasonable quality and even with max context fits perfectly within 96GB.

[nsbk]

Sweet. Are you running a Mac Studio Ultra or 4x3090?

[embedding-shape]

Simpler (and faster): a single RTX Pro 6000 :)

[androiddrew]

Curious, do you find the 3.5 120B sized MoE works better than the dense 3.6 27B?

[embedding-shape]

Yeah, Qwen3.5-122B-A10B-NVFP4 produces better responses than Qwen3.6-27B-NVFP4 (both from unsloth), but I'm mostly using them for programming in various ways, mostly Rust, Clojure, Python and JavaScript, and some translations tasks, but not much more than that, so YMMV.

Edit: as a concrete example, I'm working on a "optimization framework via agent harness" right now, Qwen3.6-27B-NVFP4 is often unable to actually complete the optimization within 100 turns, while Qwen3.5-122B-A10B-NVFP4 has no issues finishing within ~50 turns or so.

[nsbk]

Hahaha well played! Hat tip

[cmrdporcupine]

Absolutely. There's a glaring gap in the space for something about the size of Nemotron Super or just under, but actually ... competent.

The fantasy is a 100B or 80B model, but MoE and highly tuned for coding.

[pettijohn]

SO MUCH THIS. I have Strix Halo with 128GB RAM and was a large and fast model like 122B A10B. Here's hoping!

[nsbk]

Indeed! That would be the sweet spot for my 2x3090 rig

[zer0gravity]

This seems more of a battle for frontier AI supremacy. I'm afraid that small capable models have been left in the dust. Big labs don't really want to hand over the golden eggs goose to the end user. Possibly the hardware vendors(e.g. Nvidia) may want to play in that area as well, to pull money from all parties.

[ryukoposting]

> I'm afraid that small capable models have been left in the dust

I wholly disagree. Rather than going the "everything is a claude code skill" route, I've been hacking together purpose-built harnesses for all sorts of tasks, and in that environment a wee little baby model can do some really useful things. You end up burning lots of tokens making the thing, but then all that investment comes back when the resulting tool works perfectly fine on a dinky little model that fits on my 3060 Ti.

[akazantsev]

Google makes Gemma 4 31B QAT; that's not a small lab. It's one of the better models out there for consumer hardware. Allows me to run it on a 7900XTX with 64k context.

[cyanydeez]

someone will keep putting out consumer level models. Once you have the larger models, you can derive the smaller onces.

Europe will definitely be interested in democratizing these things if China starts losing interests; from there, there'll be more countries looking to keep their citizens entrained in their own Country's infrastructure.

It'll especially be true if the memory cartel keeps prices high and NVIDIA tries to gouge higher memory models.

It's an arms race everyone can join because PC hardware was mostly democratized in the last decade.

[vitalyan8184]

nvidia and amd don't give a flying fuck about end users right now while they can milk triple digit markups from infinite money VCs via data center GPUs.

[worldsavior]

That's a 2.4T model, how would they reduce this to 35B and still give some accuracy? That's a completely different arch.

[cyanydeez]

there's been a lot of research about reducing models by taking out layers; there's also using it to train smaller models by optimizing parameters.

I dont see most model building as anything more than a pig at a slop troth, despite the level of sophistication; they're still rarely pruning the input beyond random sampling.

[psychoslave]

What hardware do you have?

[nsbk]

I run a 2x 3090 rig, but a single 3090 already provides a great experience at a reasonable quant and context size. On a single card I used to run Qwen_Qwen3.6-27B-Q4_K_M or similarly quantized 35B MoE at 65536 context size

[xiconfjs]

you should look into https://github.com/noonghunna/club-3090