[planetis]

It seems to overlook that the language model was developed using a large corpora of code, which probably includes structured fuzzers for file formats such as GIF. Plus, the scope of the "unknown" format introduced is limited.

[moyix]

The original test of the GIF parser does, but the VRML parser less so and the completely novel packet parser even less so. I'm not quite sure what you mean by the scope of the "unknown" format being limited – it's not the most complex format in the world, but neither is GIF.

Another test to check how much seeing the actual parser code helps is to have it generate a GIF fuzzer without giving it the code:

https://twitter.com/moyix/status/1766135426476064774

And finally, for fun, we can see how it does when we give it the RFC for GIF89a:

https://twitter.com/moyix/status/1766207786751279298