| I'm talking about automation generally, not agent loops. E.g. prompt A to achieve X, output in format Y. Use Y to do something in prompt B. Agentic loops will underperform deterministic control flow pipelines (with non-determinism constrained to LLM calls). Agents are more general, which is the main advantage. But inherently a more general solution will waste context on unnecessary reasoning. Try asking the smaller Qwen models to output a JSON in a specific format. It basically can't do it consistently with a moderately sized prompt unless you constrain the token generation via GGML or are extremely repetitive and specific about it. (Thinking disabled) Gemma 4 will do it correctly pretty much 100% of the time. (Thinking disabled) Applies to other rule following as well in my experience. Qwen may be better at toolcalling and certainly probably codegen. It seems to me Google explicitly designed Gemma for edge device automation, and didn't fine tune for agentic or coding use cases. |