The original Zig code is metaphorically one big unsafe block. Even if the Rust port is made up of 3% unsafe blocks, that still means 97% of the original Zig code has been made safe (in the Rust sense).
The safety of safe rust is created by extremely strong and subtle invariants that unsafe rust must maintain perfectly (or all bets are off). The idea that a program with tens of thousands of lines of vibe coded unsafe Rust is memory safe is mind boggling.
IME agents are extremely happy to write more or less convincing // SAFETY: comments (most Rust code in the training set has these just before unsafe { - also it often works in unit tests). However, miri remains broadly unimpressed by comments.
Of note, even that 97% of original code that is now "safe in the rust sense" could violate invariants through safe blocks that cause memory-safety issues. I can't say exactly how the LLM-ported Rust code made use of unsafe but see https://www.ralfj.de/blog/2016/01/09/the-scope-of-unsafe.htm...
No, but when the compiler isn't enforcing anything, how do you know which ones are safe and which ones are not? If we ignore declarations etc. and only consider statement lines that read or write memory, then any such line of Zig (or C, or C++, or any other unsafe language) can be classified as a potentially unsafe — not just today, but in the future, since safety bugs can be generated upstream by giving out pointers that aren't valid or concurrency-safe.
> If we ignore declarations etc. and only consider statement lines that read or write memory, then any such line of Zig (or C, or C++, or any other unsafe language)
Surely it's obvious that's trivially untrue?
Also, literally no code is safe from potential future upstream changes.
Anyway, I'm not arguing zig has the same safety guarantees as rust. That would be silly. But the weird binary thinking around memory safety is equally silly. People want things to be simple, but you really can't come to good conclusions by trying to pretend things are simpler than they are and through a lack of analysis and thinking.
Since they did a one-to-one translation to start with, hypothetically, if these were the true numbers, you could probably get a static analysis tool to guarantee you that 97% of the zig code was safe and tell you which of the 3% were unsafe. Zig has a lot more syntax and conventions to encourage safe code compare to C after all. It just doesn’t have way to mark which parts are safe or unsafe.
An LLM could probably also trivially give you accurate reviews saying which parts were unsafe and in need of tests or reviews. I mean, considering their LLM budgets they could probably have had nightly reviews running every night for years before spending more than their Rust rewrite.
Not that I think Rust rewrite was a bad idea. Rust is a good fit for this kind of project. I say that as a Zig enthusiast. It’s just that their stated motivation and reported results are kinda BS. If they just wrote “we just like Rust and thought it’d be cool to see of LLM could do the whole rewrite”, and left it at that, I think it’d be a more honest description of the motivation. The rest is just rationalisation.
No you cannot get a static analysis tool to guarantee 97% of zig code was safe. Well, you can - the tool is called the borrow checker.
Do you know what's infinitely cheaper and faster than an LLM running nightly reviews of all of the code and infact mathematically provable that the code (and any new code) is memory safe? It's called the borrow checker.
it seems like a reasonable enterprise to intercept the data dependency graph from the zig compiler and subject it to analysis to prove memory safety parameters: