[Der_Einzige]

Secretly the problems many people have with agentic coding are related to poor choice of sampling settings, but the world will wait several more years before this is understood well. top_p and top_k are garbage but they are intentionally kept on purpose because subsequent methods enable coherent high temperature sampling, which is an absolute no go for alignment/safety reasons.

The secret to actually good agentic outputs even with small models? Llamacpp has support for this little known sampler called "top-n sigma". You should use that, set it to 1 and set temperature to literally whatever you want (it could be infinity) and your model will just magically work to your maximum context window. That's because long context generation is a sampling problem.

[major505]

interesting. using Ollama I created a simple markdown file similar do a Dockerfile creating an agent called "boring as fuck".

This is the content

FROM qwen2.5-coder:7b

PARAMETER temperature 0

SYSTEM "You are a senior software engineer focused in php and Node.js.Your responses should be strictily technicals, without poetry or prose, and focused in safety. IF you are working with legacy code, just apply the changes with the best syntax possible."