[gmaster1440]

It's refreshing to see a science fiction writer underplay the capabilities of AI, but if anyone can speak to the nuances and implications of generative AI on art and writing it's probably someone like Ted Chiang.

We can debate his generalized definition of art as making creative choices that carry subjective, intentional, and performative value for human beings (and therefore LLMs fall short of this), but I think he makes a couple strong points nonetheless:

1. The argument others like François Chollet have also made, that we have yet to see any AI systems capable of exhibiting intelligence beyond stylistic mimicry or forming generalized knowledge about concepts from large data sets.

2. The subjective experience of human interaction is valuable and desirable, and will remain so in the face of increasingly capable models, not because they won't be able to compete in producing inspiring art or enjoyable fiction, but because of the inherent primacy of human intentionality and experience.

[pessimizer]

> we have yet to see any AI systems capable of exhibiting intelligence beyond stylistic mimicry or forming generalized knowledge about concepts from large data sets.

I've said this long before useful LLMs, but I don't think we've observed this in humans, either. Human creativity can be put into two very similar categories:

1) Metaphor; the arbitrary application of the dynamics of one thing to another. "What if information is like water?" "What if the economy is like the human body?" "That woman is like a bird."

2) Bad copies. When you see someone's output and try to imitate it, but have to speculate about the creative and mechanical process that resulted in that output. You sometimes guess right and sometimes wrong, but the output is similar. Then you vary the parameters in order to create a new example, but since your process was different, with different parameters and different interactions, you create something different than the person you copied would have created.

1+2) both randomly often create emergent effects that are then copied by others, sometimes badly.

This is how Japanese metal can be the result of Black Americans copying songs from musicals and English/Irish drinking songs, British people copying the blues from Black Americans, Americans copying British Invasion music and NWOBHM, and then Japanese people copying American metal.

[gmaster1440]

Human beings are currently capable of productively searching through the space of possible knowledge and experience in ways no current AI systems are. This is not to say AI will never do this, but I think it's fair to say there are things human beings are capable of doing today that AI is not, and it remains very much unclear whether AI will ever be able to achieve important milestones like being conscious in the sense of having a subjective experience and therefore forming special knowledge that can only come from that, like Qualia.

[ben_w]

> Human beings are currently capable of productively searching through the space of possible knowledge and experience in ways no current AI systems are.

I would say the opposite: AI are very good at searching through information spaces, much better than we are.

They're terrible at learning from experience, the points made in the article about that are I think valid, but they're wildly super-human at searching.

> like Qualia

For me, the biggest issue here is: we don't know what that is, it's just what we do.

Without knowing what qualia actually is, we can't tell if an AI does or doesn't have it, we can't deliberately make a machine which does or doesn't have it.

I really hope we figure that question out before someone tries full-brain uploading/emulation.

[gmaster1440]

> I would say the opposite: AI are very good at searching through information spaces, much better than we are.

We are likely talking past each other here. By "searching" I don't mean how inference is currently carried out by efficiently analyzing the context window using weights trained on large data sets fine tuned on specific goals.

I mean the process by which novel information is discovered, which is why many proponents of AI will concede that it's not currently capable of "doing science" or making novel discoveries.

> we don't know what that is, it's just what we do.

Not sure I understand, we have a pretty good understanding of what qualia actually is, even if it can be difficult or awkward to talk about conceptually. The gap between having a subjective experience and not having one is a large one, just ask anyone who's alive but under general anaesthesia that induces loss of consciousness. Qualia is simply what arises from the quality and character of having a subjective experience.

[ben_w]

> We are likely talking past each other here. By "searching" I don't mean how inference is currently carried out by efficiently analyzing the context window using weights trained on large data sets fine tuned on specific goals.

> I mean the process by which novel information is discovered, which is why many proponents of AI will concede that it's not currently capable of "doing science" or making novel discoveries.

Huh. I think those are the same thing?

But then, I say that AI can do science. Not that I would recommend specifically an LLM for this, but what was AlphaFold doing if not science? Or even GOFAI having been used for the four colour theorem back in the day.

> Not sure I understand, we have a pretty good understanding of what qualia actually is, even if it can be difficult or awkward to talk about conceptually.

Hm. How to rephrase…

Can you create a testable definition of it?

> The gap between having a subjective experience and not having one is a large one, just ask anyone who's alive but under general anaesthesia that induces loss of consciousness. Qualia is simply what arises from the quality and character of having a subjective experience.

Purely from asking them questions, how will you differentiate between each of these cases?

1) A person under general anaesthesia that induces loss of consciousness

2) A person under the influence of a paralytic agent without anaesthesia, who is fully aware of their surroundings but unable to respond

3) A brain-dead person

4) A person with locked-in syndrome

5) A person in REM sleep who is currently dreaming but unaware of the surrounding real world

6) A person in deep (non-dream) sleep who also has no awareness of the surrounding real world

7) An unborn foetus (any species)

There are people with locked-in syndrome who later recover, who report that those around them treated them as if they were non-conscious.

[gmaster1440]

AlphaFold is really impressive and made scientific advancements and discoveries in the field of protein folding, and is now even expanding into more molecules and biology, but it was explicitly trained to do just that. You're not going to see AlphaFold write compelling science fiction.

We can build models that are specifically trained and fine tuned on scientific fields to make advancements in them, but that's different from what I'm talking about, which is building a model that forms its own hypothesis, designs its own experiments, and contributes to the wide and deep wealth of knowledge that, crucially, goes well beyond the scope of its training data.

[swayvil]

We should ask an artist if it's art. The testimony of an expert firsthand witness is vastly superior to interpretation of a secondhand abstraction by inexpert persons.

[doph]

I'd argue that an LLM has a fuller understanding of what we mean when we say "art" than most individual users of the term (Chiang's definition sounds more like the one for "craft" to me). Indeed, asking ChatGPT about the difference between art and craft gives me a more nuanced contemplation on the subject than I found in the article. Maybe these models are closer to the experts than we are willing to give them credit for? At the vet least, they are the best mirror we've ever invented.