[tossandthrow]

There is this over indexing in training data that I find quite problematic.

I have really good results getting LLMs to read documentation and work of these. This is in domains probably sparsely represented in the training data.

[docheinestages]

In my experience, even the best frontier LLMs are very likely to make critical but subtle mistakes and false assumptions the more they're trying to one-shot the solution. One-shotting could be thought of as a broad term and varies depending on the use case. You have great results with LLMs because you did the job of finding the right documentation, and more importantly, those who wrote the documentation both had a deep understanding of the domain, and effectively compiled them into a coherent document. In other words, the more vetting, supervision, and research you do, the better the results. Of coruse, this doesn't mean doing the heavy lifting yourself. But the signal is key.