[Workaccount2]

We understand and build the trellis that the LLMs "grow" on. We don't have good insight into how a fully grown LLM actually turns any specific input into any specific output. We can follow it through the network, but it's a totally senseless noisy mess.

"Cat" lights up a certain set of neurons, but then "cat" looks completely different. That is what we don't really understand.

(This is an illustrative example made for easy understanding, not something I specifically went and compared)

[EPWN3D]

We don't know the path for how a given input produces a given output, but that doesn't mean we don't know how LLMs work.

We don't and can't know with certainty which specific atoms will fission in a nuclear reactor either. But we know how nuclear fission works.

[ben_w]

We have the Navier–Stokes equations which fit on a matchbox, yet for the last 25 years there's been a US$1,000,000 prize on offer to the first person providing a solution for a specific statement of the problem:

  Prove or give a counter-example of the following statement:

  In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.

[bigyabai]

And when that prize is claimed, we'll ring the bell on AGI being found. Gentleman's agreement.

[ben_w]

I don't see how it will convince anyone: people said as much before chess, then again about Go, and are still currently disagreeing with each other if LLMs do or don't pass the Turing test.

Irregardless, this was to demonstrate by analogy that things that seem simple can actually be really hard to fully understand.

[bigyabai]

I have never once heard someone describe Stockfish as potentially AGI. Honestly I don't remember anyone making the argument with AlphaGo or even IBM Watson, either.

[ben_w]

Go back further than Stockfish — I said "people said as much before chess", as in Deep Blue versus Garry Kasparov.

Here's a quote of a translation of a quote, from the loser, about 8 years before he lost:

"""In 1989 Garry Kasparov offered some comments on chess computers in an interview with Thierry Paunin on pages 4-5 of issue 55 of Jeux & Stratégie (our translation from the French):

‘Question: ... Two top grandmasters have gone down to chess computers: Portisch against “Leonardo” and Larsen against “Deep Thought”. It is well known that you have strong views on this subject. Will a computer be world champion, one day ...?

Kasparov: Ridiculous! A machine will always remain a machine, that is to say a tool to help the player work and prepare. Never shall I be beaten by a machine! Never will a program be invented which surpasses human intelligence. And when I say intelligence, I also mean intuition and imagination. Can you see a machine writing a novel or poetry? Better still, can you imagine a machine conducting this interview instead of you? With me replying to its questions?’"""

- https://www.chesshistory.com/winter/extra/computers.html

So while it's easy for me to say today "chess != AGI", before there was an AI that could win at chess, the world's best chess player conflated being good at chess with several (all?) other things smart humans can do.