[pizza234]

I'm actually confused by the post - indeed why choosing the 27B dense and the 35B MoE models? Given the structure of the post I honestly have the impression that the Spark was purchased out of curiosity more than for a specific purpose (LLMs).

[InTheArena]

The 27b dense is _remarkably_ better at coding tasks.

Especially at larger quanitizations (Q4 is pretty crap).

[biddit]

Ah yeah, your reaction makes sense. In the circles I run in, there is a lot of hype around Sparks for inference, so my gut reaction is to respond with this type of warning.

I did not intend to imply that the post author was advocating that they're great for inference, as they're obviously not.