[simonw]

The solution to "nonexistent APIs" is to use a coding agent (Claude Code etc) that has access to tooling that lets it exercise the code it's writing.

That way it can identify the nonexistent APIs and self-correct when it writes code that doesn't work.

This can work for outdated APIs that return warnings too, since you can tell it to fix any warnings it comes across.

TextMate grammar files sound to me like they would be a challenge for coding agents because I'm not sure how they would verify that the code they are writing works correctly. ChatGPT just told me about vscode-tmgrammar-test https://www.npmjs.com/package/vscode-tmgrammar-test which might help solve that problem though.

[Razengan]

Not sure if LLMs would be suited for this, but I think an ideal AI for coding would keep a language's entire documentation and its source code (if available) in its "context" as well as live (or almost live) views on the discussion forums for that language/platform.

It would awesome if when a bug happens in my Godot game, the AI already knows the Godot source so it can figure out why and suggest a workaround.

[simonw]

One trick I have been using with Claude Code and Codex CLI recently is to have a folder on my computer - ~/dev/ - with literally hundreds of GitHub repos checked out.

Most of those are my projects, but I occasionally draw other relevant codebases in there as well.

Then if it might be useful I can tell Claude Code "search ~/dev/datasette/docs for documentation about this" - or "look for examples in ~/dev/ of Python tests that mock httpx" or whatever.

[UltraSane]

Is that much faster than having Claude Code go directly to github?

[simonw]

Yes - it can use grep etc directly and you don't have to worry about github rate limits (I hit those a lot.)

[UltraSane]

In a perfect world LLMs could generate Abstract Syntax Trees directly.