[toomanyducks]

Some of the examples given wouldn't even run, as far as I know. Copilot writing Haskell/Rust/<insert your intricate type system lang here> would probably just fail to compile.

I know very, very little about AI, but, like the OP said, it'll reproduce the traits of its training distribution. If the traits don't map on to the domain-specifics of the type ssytem, the traits don't map on, and the code fails.

Generating garbage the compiles is harder, but generating garbage in general is just as easy.