[galactushonor]

> it had of course hallucinated what the regulation actually required

Did it do the correct job once you put the regulations doc(s) in the context?

[loloquwowndueo]

What I usually do when in doubt is challenge the AI. “Please quote the section of regulation the product is non compliant with”. It usually admits it hallucinated the whole thing.

[mattmanser]

It sometimes says that even if it hasn't though, so like everything with LLMs, you can't actually rely on that.

[Chu4eeno]

You should just hit retry, usually they either latch onto the correct part of the latent space (surprisingly often) or they admit they don't know (or call a tool, depending on the model).