[Ozzie_osman]

An alternative is to view the AI agent as a new developer on your team. If existing guidance + one-shot doesn't work, revisit the documentation and guidance (ie dotMD file), see what's missing, improve it, and try again. Like telling a new engineer "actually, here is how we do this thing". The engineer learns and next time gets it right.

I don't do MCPs much because of effort and security risks. But I find the loop above really effective. The alternative (one-shot or ignore) would be like hiring someone, then if they get it wrong, telling them "I'll do it myself" (or firing them)... But to each his own (and yes, AI are not human).

[m4rtink]

I don't think you can say it learns - and that is part of the issue. Time mentoring a new colleague is well spent making the colleague grow professionally.

Time hand-holding an AI agent is wasted when all you guidance inevitably falls out of the context window and it start making the same mistakes again.

[fl7305]

> you guidance inevitably falls out of the context window

Yes, for a pure LLM session.

But using GitHub Copilot, the agent server picks out what it thinks is your most important rules, and inserts them as part of the context before running each new prompt.

So it becomes a sort of "long-term memory" outside of the context window.

You can also write your own copilot-instructions.md which is also inserted into the context window.

[Ozzie_osman]

That's why you put it in either code documentation or context files (like dotMD).

[teiferer]

> The engineer learns and next time gets it right.

Antropomorphizing LLMs like that is the path to madness. That's where all the frustration comes from.

[TeMPOraL]

On the contrary; stubborn refusal to anthropomorphize LLMs is where the frustration comes from. To a first approximation, the models are like little people on a chip; the success and failure modes are the same as with talking to people.

If you look, all the good advice and guidelines for LLMs are effectively the same as for human employees - clarity of communication, sufficient context, not distracting with bullshit, information hygiene, managing trust. There are deep reasons for that, and as a rule of thumb, treating LLMs like naive savants gives reliable intuitions for what works, and what doesn't.

[mewpmewp2]

I treat LLMs as statistics driven compression of knowledge and problem solving patterns.

If you treat it as such it is all understandable where they might fail and where you might have to guide them.

Also treat it as something that during training has been biased to produce immediate impressive results. This is why it bundles everything into single files, try catch patterns where catch will return mock data to show impressive one shot demo.

So the above you have to actively fight against, to make them prioritise scalability of the codebase and solutions.

[stavros]

Exactly this. People treat LLMs like they treat machines and then are surprised that "LLMs are bad".

The right mental model for working with LLMs is much closer to "person" than to "machine".