[opportune]

Agreed but in my opinion the problem is more fundamental than just hallucinations, it involves plain inaccuracy and inability to reason.

Try asking chatgpt or Gemini about something complex that you know all about. You’ll likely notice some inaccuracies, or thinking one related subject is more important than something else. That’s not even scratching the surface of the weird things they do in the name of “safety” like refusing to do work, paying lip service to heterodox opinions, or interjecting hidden race/gender prompts to submodels.

It’s good at generalist information retrieval to a certain degree. But it’s basically like an overconfident college sophomore majoring in all subjects. Progressing past that point requires a completely different underlying approach to AI because you can’t just model text anymore to reason about new and unknown subjects. It’s not something we can tweak and iterate into in the near term.

This same story has recurred after every single ML advance from DL, to CNN + RNN/LSTM, to transformers.

[aleph_minus_one]

> Agreed but in my opinion the problem is more fundamental than just hallucinations, it involves plain inaccuracy and inability to reason.

> Try asking chatgpt or Gemini about something complex that you know all about. You’ll likely notice some inaccuracies, or thinking one related subject is more important than something else. That’s not even scratching the surface of the weird things they do in the name of “safety” like refusing to do work, paying lip service to heterodox opinions, or interjecting hidden race/gender prompts to submodels.

For sure.

On the other hand, I recently started to convert these AI hallucinations into a feature for me: it is like asking a person who is somewhat smart, but high on some hallucinogenic drug, on their opinion on a topic of your interest. Depending on the topic and your own intellectual openness, the result can be ... interesting and inspiring.

[lex-lightning]

I’ve asked Bing about sexual health questions and been told it’s not okay to talk about that.

I asked it a question about Christianity and it stated things with the tone and certainty of a preacher.

Just gross. And worse to put something like that in the hands of billionaires.

I am somewhat of a doomer not because I think it will be the Terminator. More like AI theocracy here we go.