[pianopatrick]

At what point should I trust projects like this?

I've heard stories of LLMs changing tests to get all the tests to pass instead of actually fixing code. So I'm not sure I trust it just because tests pass. I also think that tests do not and cannot check everything.

I also don't trust it just because it compiles in Rust. The Rust compiler does not check everything.

So at what point do projects like this cross from "untrusted" to "trusted"?

[simonw]

> I've heard stories of LLMs changing tests to get all the tests to pass instead of actually fixing code.

That's very easy to prevent. Don't let them edit the tests! Run the test suite against a reserved copy.

[pianopatrick]

That makes sense to me as a tactic to prevent that problem.

But the article talks about how so much code was written so fast. Seems to me that to create that much code that fast you have to have AI produce both the code and the tests.

So I am not sure in this project if the AI can edit the tests or not. I assume that because this is Anthropic the AI is doing as much as possible, which would include editing tests.

[simonw]

Bun had an enormous existing test suite written in TypeScript. Getting those tests to pass against the Rust version was the key thing that enabled the project.

You can go and check if the AI edited the tests yourself: look at the git history of those files in the public Bun repository.

[skylurk]

But Bun also implements the interpreter to those tests— doesn't that mean it has a lot of leeway to over-fit to them? It's not as clean as "don't change these files" in this case.

In general I would agree though, and would take it one step further: the agent writing the source code should not even be able to see the tests, nor the agent writing the tests see the source code.