[dijksterhuis]

In classification problems there’s a useful term for something similar already — False Positives…

   false positive (FP), Type I error
   A test result which wrongly indicates that a particular condition or attribute is present

https://en.m.wikipedia.org/wiki/Confusion_matrix

Edit — Though I’m not sure how well that fits for a LLM (it’s more a series of false positives at each step of prediction in the sequence).

[baq]

In psychology, we've got a term which is almost 100% matching: confabulation. The only part which isn't correct is association with brain damage.

https://en.wikipedia.org/wiki/Confabulation

In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It is generally associated with certain types of brain damage (especially aneurysm in the anterior communicating artery) or a specific subset of dementias.

[NeuroCoder]

I think it's a good term for this but it also side steps the issue that this isn't a actually intelligence coming up with it. It's just machine noise

[baq]

It doesn’t matter if it’s noise, it only needs to be useful. Confabulations are not only not useful, they’re actively harmful.