[darepublic]

When I ask chatgpt to create a mermaid diagram for me it regularly will add new lines to certain labels that will break the parse. If you then feed the parse error back to it the second version is always correct And it seems to exactly know the problem. There are some other examples where it will almost always get it wrong the first time but right if nudged to correct itself. I wonder what the underlying cause is

[overfeed]

> I wonder what the underlying cause is

It responds with the statistically most probable text based on its training data, which happens to be different with the errors vs without. I suspect high-fidelity diagramming requires a different attention architecture from the common ones used in sentence-optimized models.

[stefan_]

Today I asked Claude to create me a squidward looking out the window meme and it started generating HTML & CSS to draw squidward in a style best described as "4 year old preschooler". Not quite it yet.

[qingcharles]

The issue for Claude is that Anthropic don't have an imagen that I know of, so the only tool available for the LLM to draw something is to start doing vector stuff in CSS, which is very hard for it (see the pelicans).

Gemini, ChatGPT or Grok would find this a lot easier as they could gen an image inline, although IP restrictions might bite you. Even Grok wants to lecture on IP these days, but at least it's fairly trivial to jailbreak.

[dworks]

I think the problem should be defined as "why does it not loop back the errors from the first attempt so it can fix it on the second attempt" rather than why it fails to produce a fully correct implementation on the first pass.

[karmajunkie]

you’ve got to give it a way (eg rendering with playwright and friends) and tell it to use that way to verify correctness. it’s not going to create the guard rail for you but if you provide it with one the output is much better.

[quintu5]

This is one of the issues I’ve attempted to tackle with the Mermaid Studio plugin for IntelliJ.

It provides both syntax guides and syntax/semantic analysis as MCP Tools, so you can have an agent iteratively refine diagrams with good context for patterns like multi-line text and comments (LLMs love end-of-line comments, but Mermaid.js often doesn’t).

[ar0b]

"Prompt Repetition Improves Non-Reasoning LLMs " - https://arxiv.org/pdf/2512.14982

What instance of ChatGPT are you doing that with? (Reasoning?)

[darepublic]

Observed from 5.2, on chatgpt.com. earlier versions did worse.. as in, they might take a few prompts to generate a parseable syntax. Newer versions just usually deliver one unparseable version then get it right second try. Likely I could prompt engineer to one shot but I think I would always need the specific warning about newlines.

[alex_duf]

I don't think it's about repeating the instructions, but rather providing feedback as to why it's not working.

I've noticed the same thing when creating an agentic loop, if the model outputs a syntax error, just automatically feed it back to the LLM and give it a second chance. It dramatically increases the success rate.

[deckar01]

Mermaid is really bad about cutting off text after spaces, so you have to insert <br>s everywhere. I’m guessing this is getting rendered instead of escaped by your interface. Or just lost in translation at the tokenizer.