[yawpitch]

Uh, they don’t need to understand how it works to recognize how it does not work.

> A common definition of a rational agent (human or artificial), which the authors adopted, is whether it reasons according to the rules of logic and probability.

Assuming that definition then LLMs clearly do not start from those two first principles; it is manifestly obvious that the rules of logic are not programmed into them explicitly, and since you acknowledge that LLMs cannot cope with mathematical operations (the underpinning of said rules) they also cannot have discovered either the rules of logic or probability implicitly.

Again, if relying solely on that definition (whatever the value of doing so) then it’s perfectly rational to label LLMs as irrational.

[sieszpak]

You're right. However, irrational LLMs will provide better medical and psychological advice than human experts and demonstrate much higher levels of empathy. How to combine it now - irrationally and it will replace some of the psychologist...

[yawpitch]

An LLM will give back essentially the mean advice gleaned from all the training data, which cannot be better than a human expert who has just discovered some pertinent fact not in that training data… thus if you suffer from something very rare or on the cutting edge of research, always trust the expert over the LLM.

And for much the same reason as we know an LLM isn’t rational, an LLM certainly cannot have or demonstrate any empathy, though you may well have been making a sarcastic comment about human experts there.

[mdp2021]

> better ... advice

Interspersed with lunatic delirious advice? ?!

There is a reason why people should be afraid of morons, irregardless of scholar titles.

It is incredible and very revealing how people and cultures around the world show a complete disregard and unawareness of reasoning, judgement, good "common" sense, mental sanity...