[pcwalton]

I tried to get ChatGPT to write LLVM IR last year. The results were interesting: the LLM wrote superficially correct-looking IR, but it ultimately failed to grasp the concept of SSA, as it kept trying to write to SSA registers. ChatGPT can generalize across language syntax reasonably well, but it doesn't understand deeper differences in language semantics.

[wvenable]

I tried to get ChatGPT to write 6502 assembly and it had similar issues.

[saurik]

FWIW, I had similar issues trying to get both it and Claude to help me with x86 assembly: it kept thinking if it added another * or some more parentheses it could get some impossible assembly to work.

[bongodongobob]

Reminds me of adding *'s and &'s until my C++ code worked in college.

[deathanatos]

I got this from Gemini:

> Gemini:

> JSON: In JSON, double quotes are used to delimit strings. If a double quote needs to be included within the string itself, it's escaped by doubling it (e.g., `"This is a string with a "quote" in it."`).

(And, just an editorial note: there are no backslashes in Gemin's response / HN is displaying it correctly. Gemini's response is nonsense — wrong, but also not even internally consistent.)

> Me:

> Does your JSON example parse as valid JSON?

> Gemini:

> Yes, the JSON example I provided ("This is a string with a "quote" in it.") is actually valid JSON.

> The reason it's valid is because the doubled quote within the string is interpreted as an escaped character, representing a single literal quote within the string itself. The JSON parser recognizes the double quotes at the beginning and end as delimiters for the entire string, not the escaped quote within.

(Its example, is, of course, quite malformed.)