[dgroshev]

It's not just things that can be caught with tests. You wouldn't know if you got recommended an obsolete API, a dated pattern, or a charged foot gun, and if you're looking up docs for every suggestion I doubt that'd save you much time compared to reading docs through to begin with.

[jebarker]

I can definitely see those things being problems, but they are much more likely to occur when asking an LLM to generate code for you. That's not the best coding use case for LLMs IMO, it's much better to provide it code already written and ask it for explanations, bug fixes, refactoring etc. Then it rarely introduces new API calls etc