[ben_w]

> Human beings are currently capable of productively searching through the space of possible knowledge and experience in ways no current AI systems are.

I would say the opposite: AI are very good at searching through information spaces, much better than we are.

They're terrible at learning from experience, the points made in the article about that are I think valid, but they're wildly super-human at searching.

> like Qualia

For me, the biggest issue here is: we don't know what that is, it's just what we do.

Without knowing what qualia actually is, we can't tell if an AI does or doesn't have it, we can't deliberately make a machine which does or doesn't have it.

I really hope we figure that question out before someone tries full-brain uploading/emulation.

[gmaster1440]

> I would say the opposite: AI are very good at searching through information spaces, much better than we are.

We are likely talking past each other here. By "searching" I don't mean how inference is currently carried out by efficiently analyzing the context window using weights trained on large data sets fine tuned on specific goals.

I mean the process by which novel information is discovered, which is why many proponents of AI will concede that it's not currently capable of "doing science" or making novel discoveries.

> we don't know what that is, it's just what we do.

Not sure I understand, we have a pretty good understanding of what qualia actually is, even if it can be difficult or awkward to talk about conceptually. The gap between having a subjective experience and not having one is a large one, just ask anyone who's alive but under general anaesthesia that induces loss of consciousness. Qualia is simply what arises from the quality and character of having a subjective experience.

[ben_w]

> We are likely talking past each other here. By "searching" I don't mean how inference is currently carried out by efficiently analyzing the context window using weights trained on large data sets fine tuned on specific goals.

> I mean the process by which novel information is discovered, which is why many proponents of AI will concede that it's not currently capable of "doing science" or making novel discoveries.

Huh. I think those are the same thing?

But then, I say that AI can do science. Not that I would recommend specifically an LLM for this, but what was AlphaFold doing if not science? Or even GOFAI having been used for the four colour theorem back in the day.

> Not sure I understand, we have a pretty good understanding of what qualia actually is, even if it can be difficult or awkward to talk about conceptually.

Hm. How to rephrase…

Can you create a testable definition of it?

> The gap between having a subjective experience and not having one is a large one, just ask anyone who's alive but under general anaesthesia that induces loss of consciousness. Qualia is simply what arises from the quality and character of having a subjective experience.

Purely from asking them questions, how will you differentiate between each of these cases?

1) A person under general anaesthesia that induces loss of consciousness

2) A person under the influence of a paralytic agent without anaesthesia, who is fully aware of their surroundings but unable to respond

3) A brain-dead person

4) A person with locked-in syndrome

5) A person in REM sleep who is currently dreaming but unaware of the surrounding real world

6) A person in deep (non-dream) sleep who also has no awareness of the surrounding real world

7) An unborn foetus (any species)

There are people with locked-in syndrome who later recover, who report that those around them treated them as if they were non-conscious.

[gmaster1440]

AlphaFold is really impressive and made scientific advancements and discoveries in the field of protein folding, and is now even expanding into more molecules and biology, but it was explicitly trained to do just that. You're not going to see AlphaFold write compelling science fiction.

We can build models that are specifically trained and fine tuned on scientific fields to make advancements in them, but that's different from what I'm talking about, which is building a model that forms its own hypothesis, designs its own experiments, and contributes to the wide and deep wealth of knowledge that, crucially, goes well beyond the scope of its training data.

[ben_w]

Ah, sorry, I was editing while you responded.

> You're not going to see AlphaFold write compelling science fiction.

Yes? But not many biology PhDs do that either.

> I'm talking about, which is building a model that forms its own hypothesis, designs its own experiments, and contributes to the wide and deep wealth of knowledge that, crucially, goes well beyond the scope of its training data.

A few weeks ago we saw LLMs do the first half of that. I think AlphaFold demonstrates the last.

Don't get me wrong, I trust the person who told me these are not good papers, but it does do those things: https://sakana.ai/ai-scientist/

[gmaster1440]

I'm happy to leave the conversation here. I don't necessarily disagree with what you're saying, but we appear to be making different points, or at least at different levels of description, and it's not really productive anymore.

[ben_w]

Certainly, my confusion would be compatible with that.

Have a good weekend :)