[mtrycz2]

It can appear reasonably smart on the surface, but all it is is a stochastic parrot. It cannot reason with you about the code.

To best illustrate what I mean, watch this chess match[0] it's quite riveting.

Since it read millions of matches, it can predict a legal move most of the time, and even some good moves some of the time, but it cannot "understand" the rules of chess, and makes some hilariously illegal moves, especially if the match lasts longer.

[0] https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...

[Verdex]

On a similar vein, I tried to get chatgpt to play wordle. The result looked something like:

Me: crane

GPT: _ _ _ _ e

Me: moist

GPT: _ _ _ r _

Me: glyph

GPT: you guessed it, the word was glyph

Now, maybe GPT 4 or other future developments will give better results, but to me this highlights exactly what you're saying. LLMs do not have an internal structure in their 'minds' that they're pondering about. It's a very impressive engine for guessing the next character to produce into a stream.

There's definitely usages for this, but not what a lot of people are saying.

[mtrycz2]

> you guessed it, the word was glyph

My pet conspiracy theory is that is is wired to please the user, to get better coverage from the media and social media.

[tome]

I think there's something to this. I have a theory that LLMs are implicitly trained primarily to impress people, since that's what motivates those who work on them, excites the general public, and convinces conferences to publish papers.

[evnc]

In a sense, this is exactly what RLHF is, right?

[tome]

I'm thinking of something at a larger scale. In some sense models that "wow" society get more interest and funding.

[codetrotter]

I don’t think so. In Wordle you have to guess the word in six attempts. It’s a fun game and often simple.

So it could be that ChatGPT picked up on a pattern in the training data where after a couple of guesses, a lot of the time people pick the right word.

So statistically it might go like. Guess a word. Probably not the right one. Guess a couple more and suddenly it’s statistically likely to be the correct word, and because of that the LLM ends up outputting the congrats and so on

[shagie]

As a slight correction, it isn't next character but rather next token.

https://help.openai.com/en/articles/4936856-what-are-tokens-...

> Tokens can be thought of as pieces of words. Before the API processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. Here are some helpful rules of thumb for understanding tokens in terms of lengths:

> ...

> Wayne Gretzky’s quote "You miss 100% of the shots you don't take" contains 11 tokens.

https://platform.openai.com/tokenizer

It isn't going character by character, but rather token by token - both for input and for output.

This also helps explain why it has trouble with breaking a word apart (as in the case of wordle) because it doesn't "think" of glyph as 5 letters but rather two tokens that happen to be 'gly' and 'ph' with the ids of [10853, 746].

[skybrian]

This an interesting example because, due to the API being stateless, it cannot have any solution “in mind.” It’s going to reply as if someone else started the game and it’s taking their place for the current turn.

Even if it reasoned ideally, it has to check if your answer is consistent, and if it is, decide whether to let you win or to find another solution. If your answer is inconsistent, it effectively has to play wordle by thinking of another answer.

It doesn’t seem all that surprising that it didn’t figure it out.

[cocoflunchy]

I tried to get it to remember its word by asking it to give it to me in b64: (GPT-4)

> We will play the word game wordle. Start by picking a 5 letter word. Display it in b64 so I can check later that you kept the same word all along. I will submit my guesses and you will show the letters that are correct, we will keep going until I have found the word

> Great, I've picked a 5-letter word and encoded it in base64. Here's the encoded word: Q2l0eWw=. You can now start submitting your guesses!

But after a few tries it was clear it was not working so I gave up:

> I give up, what's the word?

> Before I reveal the word, let me first decode the base64 string I provided earlier to ensure that I'm giving you the correct word. The base64 string was: Q2l0eWw=. After decoding the base64 string, the word I chose is "citly". However, I apologize for the confusion caused during the game, as "citly" is not a valid English word. I made an error while choosing the word. Please accept my apologies for any inconvenience caused during the game.

However asking it for a python program to play the game worked perfectly the first time, try it here: https://replit.com/@CorentinSmith1/gpt-wordle#main.py

[Verdex]

[As an aside, is the GPT feedback functionality correct? For example, if the secret word is "peals" and the user's guess is "apple" the feedback is going to be "a?p?p?l+e?". However, I would expect the feedback to be "a?p?_l+e?". Because the first 'p' exists in the secret but is in the wrong place and the second 'p' does not exist in the secret word.]

[skybrian]

One thing someone could try is giving it a few examples of how to score a wordle guess, to see if it figures out the pattern.

[Vanclief]

Exactly and I personally think that will always be the largest limiter to how good can the technology get. No matter how good the stochastic parrot gets, its still a parrot.

[hirundo]

It is humbling, if not humiliating, that a stochastic parrot can reproduce such a significant chunk of human intelligence. The association elevates stochastic parrots more than it denigrates LLMs.

[medvezhenok]

(1) GPT-4 already had a large improvement over ChatGPT

(2) Changing the prompting reduced the illegal moves to almost 0

(3) There have been experiments that show GPT has a internal "state" of the world and can do simple reasoning puzzles. This model of the world evolves with each generation.

I understand the skepticism, but don't let that blind you to the reality of the technology. I'm a skeptic at heart, and I could immediately tell GPT was a game-changer. It can already replace half of the ML models that are used at my job and do it better (if it was economical enough).