[kikimora]

Nobody reviewed resulting code. Maybe all tests are empty and this is why they pass. Maybe tests were modified to pass because this is the only thing LLM could do to make them pass. Maybe it hallucinated something in the process. We have no idea.

[jsnell]

We do have an idea, and it contradicts your guess: https://news.ycombinator.com/item?id=48133806

[kikimora]

You suggest there is only JS tests that do not need a rewrite? This is crazier than I thought…

I took tests as an example. There are so many other things that can go wrong. Rust and Zig standard libraries may have different semantics not picked up by AI. Like one guarantees insertion order of a dictionary and other does not. Differences in how runtimes react to Linux signals, how they do file IO, etc.

If I were a Bun user I would be moving off from bun unless it has excellent test coverage (which I think it does not). During a normal release cycle I offered a small increment of functionality with small number of issues. Here I’ve been offered a complete rewrite, potentially having thousands of issues. I don’t want to be a guinea pig in this experiment.

I’m genuine curious how this will unfold.