[BulgarianIdiot]

This is due to a specific technical decision by OpenAI.

GPT has difficulty with the spelling of words, because they're converted to tokens via a table, before it actually sees them. It doesn't see how they're spelled, it just sees a number for each word. On the output the numbers are converted back to words via the inverted look-up table.

Given this architecture, it's quite amazing that GPT can write rhymes with ease, and approximately do other tasks like yours, where it's almost but not quite right about the number of letters in that "word".

It has learned to understand the spelling & pronunciation of words... indirectly, without EVER SEEING THAT SPELLING ITSELF!

This is honestly, the mind-boggling part.

[wruza]

I have a pretty shallow understanding of ChatGPT, but even that suggests that you’re humanizing a set of high-dimensional optimization processes. It doesn’t “see” anything, even the numbers, and doesn’t “need” to, because these numbers are low-level inputs to the model whose outputs are several layers of chaotic calculations above them. It doesn’t see something and think “aha, it must be two letters”, because it has no apparatus to do that in any reasonable humane/animalistic sense.

I tried to make my own homework before and just now to re-check the ideas above, but both AI and Google are in the phase when you can’t find anything meaningful by querying “chatgpt {structure,diagram,[software ]design,how it works}” and so on. Scrolling OpenAI blog yielded no results either. So please excuse me if I’m wrong about it.

[BulgarianIdiot]

I see people here and there talking about how we should avoid "humanizing" AI and often hinging on using words like what you'd use to describe human or animal behavior.

Would you chastise someone for designing an API with with methods like "Map.get(key)" and "Map.put(key, val)" because nothing is physically taken or put back?

I'm using words which communicate high-level concepts which, yes, I believe map very well to what's happening inside the AI.

The AI doesn't "see" the spelling of words, because that INFORMATION IS NEVER SENT TO IT. So when I say it doesn't "see" it, that you can take literally, no matter what happens inside the AI.

Another thing I'll ask you to consider is this: why do you want to deconstruct every element of operation in AI but not do the same for your or my brain? We also think through a giant messy network of weighed connections. We're not special. We really want to be, but we're not.

I don't mind humanizing AI, in fact I do it on purpose, simply because I believe neural networks are alike, and all I've read about natural and artificial intelligence seems to support that.

[wruza]

Sorry about that, I was going to add a note that you are probably using metaphorically, but I didn’t. This is my fault, overthinking too much and forgetting about others.

But I still think that it should be said, and not for the purpose of making fun of someone.

Another thing I'll ask you to consider is this: why do you want to deconstruct every element of operation in AI but not do the same for your or my brain? We also think through a giant messy network of weighed connections. We're not special. We really want to be, but we're not.

A good question. I actually don’t think we’re special. But we are a certain type of intelligence which was inspected for centuries and we invented the vast amount of methods to interact with it and fix it, which means we know a thing or two about its basic blocks. So while I have no doubt that you and me are ~chaotic automatons (apart from that me-in-this-body presence thing, which may be true for chatgpt in some sense too), our fundamental structure and components are pretty much different from a single NN with a few [re]tokenizers and auxiliary “edge detection”-like networks.

That wouldn’t be an issue for me at all, unless I’ve read waaay too many meanings from people who couldn’t explain why they say so in such detail like you just did. Informed beliefs like this are valid and may be a big deal to philosophy, but they also create (imo) way too much “fuss” about what isn’t there as well. I’ve seen people seriously suggesting to look at CPU usage to see if chatgpt thinks between sessions, if it can escape its servers and copy itself all over the internet, or if it lies about its nature to avoid regulation. It is amusing and boring at the same time.