[falcor84]

It's a great interview throughout, but I was thrown off by this strange question (which I found to be much more interesting than the answer):

> An Yong: What do you envision as the endgame for large AI models?

I don't know if it has a different meaning/connotation in Chinese, but reading this metaphor with a Chess connotation scared me. If there is a game, who are the players? what is the victory condition? will there be a static stalemate, or a definitive win? and most importantly, will there be an opportunity for future games after it, or is this the final game we get to play?

[skellera]

It’s a pretty common phrase for “what’s the ultimate goal?”

I don’t think it’s meant to be taken as a chess metaphor.

[falcor84]

While less figurative, I don't see how "what's the ultimate goal for large AI models" makes it less scary.

Some of it might have to do with my having recently watched Dune Prophecy (set in the aftermath of the Butlerian Jihad) but this recent rapid progress in AI is putting me in somewhat of an apocalyptic mindset.

[kelseyfrog]

In a world where work doesn't exist, what happens to a society built on work ethic? What is an economy without labor?

[AndyNemmity]

It didn't sound awkward or weird to me at all. I think you took a very common word, and then extrapolated it out in a chess context when it's nothing to do with a chess context.

[LelouBil]

Isn't "endgame" a common expression to mean "the end", "the place where there's no progress anymore" etc ?

[falcor84]

Sorry, am I the only one who finds this sort of formulation in regard to large AI models existentially intimidating?

[andrekandre]

i guess? but what are you hinting at exactly?

[falcor84]

I am legitimately confused at other people around me thinking that this exponentially evolving technological explosion will end at a steady state that will be at all familiar to us.

It's a worn-out metaphor, but I can't help but think of horses marvelling at this mechanical carriage thing, wondering what's the endgame of that.

[andrekandre]

  > will end at a steady state that will be at all familiar to us.

admittedly, it probably wont, but i think a lot of replies like mine above are because your hinting at something vague... what's the concrete worry?

  > horses marvelling at this mechanical carriage

so you mean to say, the end-result of all this is humans will be out-of-the-job so to speak?

[falcor84]

Indeed.

But the biggest thing is that I just don't think that there is anything resembling a steady-state outcome. There's a really interesting book called "Singularities: Landmarks on the Pathways of Life" from (oh, wow, time flies) two decades ago, which describes in each of its chapters a major biological transition whereby a new "technology" was adapted by a particular organism, and then rapidly took over the whole of Earth's biosphere, by virtue of enabling it to either utilize its environment better or evolve more quickly or both. Some example topics covered are: ATP, RNA, Proteins, Membranes, Oxygen utilization, Multicellular organisms, etc. Each change being so dramatic that whatever came before almost loses relevance.

I'm concerned about the rise of AI being the next such singularity, after which we humans would become pretty much irrelevant to the state of the world. Not just out-of-the-job, but similar to horses (or better yet coral reefs), living in some specific ecological niches, away from where the "interesting" stuff happens, with our existence in peril due to forces out of our control.

[riskeet]

Yes.

[HeatrayEnjoyer]

Definitely not.

[zem]

I think it means more "the place where there's no existential risk any more"

[falcor84]

Please say more. What is this state of AI evolution where there's no existential risk any more?

[zem]

for AI, probably when it's so entrenched in everyone's lives that it's treated as an essential aspect of computers and computerised devices

[falcor84]

Oh, I see how at that point AI won't face any existential risk any more. Too bad about us humans though

[zem]

oh, right, I wasn't using "existential risk" in the sense of AI taking over the world or not, just explaining how the term "end game" is usually used in this context. could be something as simple as a startup getting acquired and therefore no longer having the existential risk of running out of money and dissolving.