[j1elo]

If ChatGPT is just a glorified autocomplete engine, which doesn't know the context or the meaning of what is being said (as I've read in random discussions here on HN), then how come it is able to process the license text and summarize it in those terms? I would think that it would be needed to "understand" the license text in order to produce that easier to understand version.

So, does chatGPT really know what it's writing about, or not?

[nr2x]

A lot of human “decision making” literally happens after some pattern recognition system in your brain makes the decisions. We are all just autocompleting all day.

Does ChatGPT “think”? Maybe? Do we “think” as much as we think we do? Maybe not?

https://www.psychologytoday.com/us/blog/unconscious-branding...

https://www.nature.com/articles/news.2008.751

[esafak]

We are "glorified" autocomplete engines too. The glory is in the fact that we have rich sensors, a memory, and a world model underpinning the engine. ChatGPT has a complex model, if not as complex as ours, and it is good enough to perform such tasks. As these models improve, they will compete with and outperform us on more and more tasks, at which point we will stop comparing our abilities to execute tasks.

"Sure, it can do everything, but it's not human."

[acchow]

> We are "glorified" autocomplete engines too

This doesn’t feel intuitive to me or convincing. What evidence do you have to support such a claim? I certainly don’t feel like I’m an autocomplete engine.

Was Gödel also an autocomplete engine? How could an autocomplete engine come up with novel ideas like the incompleteness theorems and understand them?

[thewataccount]

IDK if it helps but I find these examples interesting, predict the most likely word for the following phrases (whatever immediately comes to mind first):

- hello! How are

- What is 1+1?

- Today is Monday. That means yesterday was

- My favorite color is

Humans will likely estimate these as "you", "2", and "sunday". For the last one, there's not enough context to estimate the next word, but we will almost certainly estimate it as a "color", and might just have to guess at random "red"/"blue"/"orange"/etc, so there can be multiple possible answers. If you've seen enough data you might know that a specific color like blue might be more frequently mentioned in this context, and would be more included to guess "blue" because of that.

1+1 is interesting because most humans can immediately estimate 2. However given the question "What is (3.21^7)/(3+6.4*11)?" we can no longer quickly estimate the answer, we would have to manually (even if mentally) calculate the answer. I think this closely parallels how our current LLMs can do basic math they've seen a lot, but fall apart on more complex math since they aren't able to do the actual calculations and are forced to estimate.

"Today is Monday. That means yesterday was" is interesting because this would appear to imply that there is "logic". That you listed out the days of the week, looked at monday, and chose the day before. However, most likely you "just knew" it was most likely sunday given your previous knowledge. The LLM is unable to actually compute the answer, but it can estimate it based on it's training data.

[lxgr]

> I certainly don’t feel like I’m an autocomplete engine.

Your mind is what you think and feel with, so I'd take that feeling with a grain of salt on multiple levels.

> Was Gödel also an autocomplete engine? How could an autocomplete engine come up with novel ideas like the incompleteness theorems and understand them?

I think you're reading too much value judgement into the (factual, if provocative) claim of "being an autocompletion engine" here.

[codethief]

> How could an autocomplete engine come up with novel ideas like the incompleteness theorems and understand them?

Depends on your definition of "novel". Ideas like the incompleteness theorems, General Relativity, quantum mechanics, etc. were certainly new/unusual given the status quo in science at the time but they weren't coming out of thin air, either[0]. In fact, could it be that they were inevitable, given the data? Now you're no longer that far away from auto-completion.

[0]: We always ascribe to Einstein this singular genius (which he certainly was in a lot of ways) but one can easily forget over this that there's a clear survivorship bias – many other very smart people were working on unifying gravity with Special Relativity at that time.

[maushu]

When people say that humans are "autocomplete engines" they are being a little facetious on the fact that we are pattern recognizing machines that require sensory input to function. Basically "autocompleting" the input we receive.

[flangola7]

>How could an autocomplete engine come up with novel ideas like the incompleteness theorems and understand them?

Increase the temperature to above 0.

[acchow]

Sure, you can create new sentences. But you can't recognize which ones are "true" and which are not.

[thewataccount]

Neither can we. We have a lot of theorems ourselves that exist because they are what we consider the most likely solution to either extrapolating existing rules or modeling a set of measurements - but it's unknown as to whether the theory is absolutely true.

a child might see that

1 + 2 = 2 + 1

4 * 3 = 4 * 3

7 + 7 = 7 + 7

They might predict this may apply elsewhere, and that:

8 + 0 = 0 + 8

This is the Commutative Law, which the child can predict exists, even if they don't have the specialty to write proofs and understand the theory, etc. They don't know that this is true, but based on their observations this is a theory that holds true to this situation.

[acchow]

Sure, children can pattern match. But I was thinking more about seasoned mathematicians, who not only create new theorems but can also understand proofs and thus find errors in proofs and decide they are wrong, or can be correct. They can also construct new mathematical structures, which other people may eventually use in novel ways to solve old problems. None of this seems anything remotely like "pattern matching" in any way.

I don't disagree with the notion that "probably 80% of Americans/humans are just glorified autocomplete engines", but what about the people with proper ingenuity that is either provably correct in the mathematical sense, or can build up a tapestry of understanding from which we can build predictions about reality (in the physics sense).

Pretty much everything else I do in life - riding a bike, cooking food, composing/enjoying music, etc - can be done with estimates and pattern matching. But the "correctness" of math proofs seems to me to not fit the approximation/estimations that language models use.

[esafak]

It's the "intelligence is compression/prediction" paradigm, which is well known in the field, if you want to read about it.

[shagie]

The question then is "what are you?"

Our Storytelling Minds: Do We Ever Really Know What's Going on Inside? - https://blogs.scientificamerican.com/literally-psyched/our-s...

There's a school of thought that says that "you" (the thing that says "I" when you think) is part of your brain that is making up an ongoing story about the input that it sees from one instant to the next to try to make it into a cohesive story/timeline that is fed back into itself to give it continuity.

This can be seen when there's a severed brain (done to control epilepsy) so that the two halves of the brain are no longer able to communicate... but one half still sees with one eye and the other half sees with the other... and one half has control of one hand and the other half has control of the other hand... and one half can control what is said.

--

So... what are you?

Your Brain Is a ‘Prediction Machine’: It Predicts What the Other Person Is Going to Say - https://www.learning-mind.com/your-brain-is-a-prediction-mac...

You Took the Words Right Out of My Brain https://www.nyu.edu/about/news-publications/news/2014/april/...

The "get the full input, process it, and do something" isn't fast enough for how you think. Your brain is trying to predict what is happening before it happens. Goalies in hokey and batters in baseball are examples of this - they are acting before a full set of information can be processed. They are predicting the future state of the world.

With words, two people can get into the same mental state and be able to predict the word that the other will say next. Twins, spouses... and even strangers in the right circumstance can have this happen. We are "next word predictors" in that sense - and there have been experiments that demonstrate that (the two articles on the same one above).

---

And so while the story that you tell yourself about how you think doesn't match a next word predictor (and mine doesn't either) - that is a model of how the mind and brain work that has... well, predictive power.

The interesting part with GPT (to me) is that as this "next word prediction" gained more abilities to properly predict the next word it also gained a world model.

On the left is GPT-3.5. On the right is GPT-4 :: https://twitter.com/d_feldman/status/1636955260680847361 - https://news.ycombinator.com/item?id=35214218

The question of "is there a world model that is contained within the language model that can be used?" is a rather interesting one. I find the image, prompt and response set with the balloons and "what would happen if the strings were cut?" to be rather fascinating.

https://youtu.be/7VSWyghVZIg also goes into this.

I believe you're getting too hung up on the "word" of "next word prediction". But instead consider the question: "does language enable thought?" Consider examples such as Koko the gorilla, Bunny the dog and Billie the cat. Do they think? Were they able to think before they had language? If you've got a program that can manipulate language as convincingly as a human - can it think for some definition of the word "think"?

[api]

A lot of these questions hinge on words like “think” and “understand” for which we do not have precise definitions. It’s really kind of calling bullshit on a lot of the terminology we use to talk about thinking, revealing that most of it is hand wavey anthropocentrism.

It does what it does. We do not know what that means. We do not really know how close what it is doing is to what we do.

It’s also possible that it’s doing similar things to what we do but in some very different way. “Neural network” has always been a term to make biologists cringe as they’re really only the loosest of analogies to how brains work.

[xp84]

So true! We basically define those words almost self-referentially. Understanding is what human brains do, so by definition nothing else but a human can 'understand' complex things (we say).

We make only slight allowances for animals, allowing that they can "understand" but only simple things.

As this tech gets better, those who cling to that definition are going to get more and more uncomfortable, and tie themselves into more and more logical knots trying desperately to explain why it "doesn't count."

[lxgr]

A similar thing has happened (and arguably is still happening) when it comes to animal cognition.

"The only species capable of recognizing itself in the mirror. Ok, recognizing itself in the mirror and object permanence. Ok, all of that and mourning its dead. Ok, hang on for a minute..."

[matrix_overload]

Because it can instantly go through countless examples where people were summarizing things on the forums, paper abstracts, documents, etc., and blend it together with a 99% accuracy. Except for that 1% where it will look totally plausible, but will be completely wrong.

[sangnoir]

> I would think that it would be needed to "understand" the license text in order to produce that easier to understand version.

Have you interacted with Markov chains before? They are surprisingly coherent and how they work is easier to grok (or implement as a weekend project)

[buggyipadmettoo]

I was wondering the same and asked gpt to explain it to me the other day. Basically it’s all one process, there aren’t 2 modules (understand and generate). The “understanding” is perhaps like the generating in reverse. Rather than generating the next word from a statistical process, it’s defining the statistical process from the next words of the input.

But as I explain here, even reviewing that conversation we had, I think maybe gpt and I were both hallucinating and feigning understanding. Cuz I still don’t get it. It still seems far too magical to fit this explanation.

So my next best guess: we’re a simulation, GPT is magic, and our creators are just having fun introducing more and more magic into the system.

[_a_a_a_]

> I would think that it would be needed to "understand" the license text in order to produce that easier to understand version.

well IMO it did, so it does. So yes, it 'understands' by your predicate.

[andrewstuart2]

The words it autocompleted were the most likely words (chains of word fragments) to come next in a theoretical conversation when the question as context was taken into consideration, given the rest of the context it's been trained on up to EOY 2021.

[vonnieda]

I wonder what an LLM would say if it had, say, 2 orders of magnitude more parameters and 46 years of training like I do. And if it was trained up to the millisecond with new inputs from millions of individual sensors. It might say:

> I wonder what an LLM would say if it had...

[dylan604]

it'll be funny to find some unheard of website where someone wrote something very similar in blog or reply in comment. maybe even in a usenet chat archive or something?