One would think that if you asked it to help you make drugs you'd want hallucination as an outcome.
But no. Only a very, very small percentage of drug users want hallucinations.
Hallucinations happen usually, when something went bad.
(So a hallucinating LLM giving drug advice might as well result in real hallucination of the user, but also a permanent kidney damage)
One would think that if you asked it to help you make drugs you'd want hallucination as an outcome.