[TheGoddessInari]

IMO it's the same problem with Rust. LLMs are trained on vast quantities of code that violate Rust soundness in safe code.

LLMs are prediction engines: you can move the needle (heavily) on the predictions from lazy to correct by construction, but the defaults even on high end models like Opus are always riddled with shortcuts.

LLMs can produce coherent & safe Rust 1.92, LLMs can produce coherent Zig 0.16, but that's dependent on in context learning & instruction following.

[skeledrew]

> that's dependent on in context learning & instruction following

Not just that. For Rust in particular, since unsafe code is uncompilable code (unless explicitly marked "unsafe"), the LLM is essentially forced to continue iterating even if the code already works but is unsafe, until it finds an implementation that's both correct (per the spec) and safe (per the compiler). That's the difference from Zig, where "make the code safe" is essentially part of the spec, and so may be missed by the LLM and not be discovered by a human until there's an observable runtime issue, or CVE.

[maleldil]

> LLMs are trained on vast quantities of code that violate Rust soundness in safe code.

Why is your reasoning behind this? Most Rust code in the wild does not use unsafe. Most projects don't have a single line of unsafe.