[Areena_28]

I know even we hit the same thing building internal security tooling. our model kept formatting output like documentation, not like how we would or any person in place of us would read in a terminal at 2am during an incident.

I am a bit curious, did you find this behavior consistent across models or is it more pronounced with certain ones?

[noemit]

I ran into it while building - I should have tested different temps too - I was just trying to get cli style tool calls to be more reliable

[Areena_28]

yeah temperature is probably worth a run, we noticed even small adjustments changed how the model interpreted formatting expectations quite a bit.

[stuaxo]

Literate programming is about to become mainstream in the funniest way possible.

[Areena_28]

oh yesss, except literate programming was still the human explaining intent to other humans. this is more like the human explaining intent to a machine that then explains it back to other humans. hahaha this is actually funny.