[phoehne]

It doesn’t really have a model for choosing. It’s closer to pattern matching. Essentially the pattern is encoded in the training of the networks. So your query most closely matches the stuff about X, where there’s a lot of good quality training data for X. If you want Y, which is novel or rarely used, the quality of the answers varies.

Not to say they’re nothing more than pattern matching. It’s also synthesizing the output, but it’s based on something akin to the most likely surrounding text. It’s still incredibly impressive and useful, but it’s not really making any kind of decision any more than a parrot makes a decision when it repeats human speech.

[pixl97]

>If you want Y, which is novel or rarely used, the quality of the answers varies.

Is this any really different than asking a group of humans about the novel and measuring the quality?

[haldujai]

Couple differences:

1. Humans aren’t entirely probabilistic, they are able to recognize and admit when they don’t know something and can employ reasoning and information retrieval. We also apply sanity checks to our output, which as of yet has not been implemented in an LLM. As an example in the medical field, it is common to say “I don’t know” and refer to an expert or check resources as appropriate. In their current implementations LLMs are just spewing out BS with confidence.

2. Humans use more than language to learn and understand in the real world. As an example a physician seeing the patient develops a “clinical gestalt” over their practice and how a patient looks (aka “general appearance”, “in extremis”) and the sounds they make (e.g. agonal breathing) alert you that something is seriously wrong before you even begin to converse with the patient. Conversely someone casually eating Doritos with a chief complaint of acute abdominal pain is almost certainly not seriously ill. This is all missed in a LLM.

[pixl97]

>. Humans aren’t entirely probabilistic, they are able to recognize and admit when they don’t know something

Humans can be taught this. They can also be taught the opposite that not knowing something or that changing your mind is bad. Just observe the behavior of some politicians.

>Humans use more than language to learn and understand in the real world.

And this I completely agree with. There is a body/mind feedback loop that AI will, be limited by not having, at least for some time. I don't think LLMs are a general intelligence, at least for how we define intelligence at this point. AGI will have to include instrumentation to interact with and get feedback from the reality it exists in to cross partial intelligence to at or above human intelligence level. Simply put our interaction with the physics of reality cuts out a lot of the bullshit that can exist in a simulated model.

[phoehne]

Only when you’re asking for a memorized response. If you were at ask me to create a driver for a novel hardware device in Ada, there’s no memorized answers. I would have to work it out. I do that by creating mental models, which LLM’s don’t really have. It has a statistical encoding over the language space. Essentially, memorization.