[4death4]

What makes you think that? What we can LLMs are a specific architecture that literally does predict the next token, one at a time. This isn’t the only way to generate text. Perhaps humans use a method that synthesizes all the text at once. Or some other, unimaginable way.

[FrustratedMonky]

Not sure about everyone else, but I write one word at a time.

[taeric]

Almost certainly not true. Consider pronouns, as they reference other words, therefore cannot be written in isolation. Or simple rules about when to use "an" or "a".

Shaping your words to sound correct is very common. Both in speech and in writing. Sometimes it is finding how to fit a word you want to use into a sentence. Sometimes it is building a rhyme.

You may feel that you go a word at a time, but that really shows how embedded language is.

[FrustratedMonky]

Right now we don't know if LLM's are also doing this, or not.

In their calculating the 'next' word, as part of that 'weighting', are the 'simple rules' for future words, that you are saying humans do but LLM's can't.

[4death4]

It doesn't seem LLMs are capable of "understanding" that they don't "know" something, so there are certainly some observable differences. But the intent of my original comment was precisely to highlight that we don't know. It could be that next token predication is somehow mathematically equivalent to what we call consciousness. That would certainly be a revelation.

[taeric]

Ah, fair. I was only commenting on the idea that you write one word at a time. Seems fairly unlikely to me. You expand out ideas.

[FrustratedMonky]

LOL. Yes, started as a joke.

[ben_w]

I seem to have two parts of my inner monologue, one comes up with complete concepts, the other puts them into words. When I started to notice this, I tried skipping the "make words" part to save time, as clearly I had not only had the thought but also was aware that I had already had the thought. This felt wrong in a way I have no words to describe as there's nothing else like that particular feeling of wrongness, though I can analogise it as being almost but not quite entirely unlike annoyance.

Anyway, point is 80% of this comment was already in my head before I started typing; the other 20% was light editing and an H2G2 reference.

[FrustratedMonky]

Sounds kind of like some Zen concepts about moving beyond language. And that language is a faulty method for communicating.

[dartos]

Yeah but do you write that one word based on the past 1000 or so words that come before it and how each of those words relate to each other with a perfect recollection of each word at the same time?

And if you believe you do that subconsciously, then how can you be sure you don’t subconsciously plan a few words ahead?

[FrustratedMonky]

I don't think I do. To a large degree, GPT does write better.

[dartos]

Write better as is has more correct and clear syntax, sure.

But it doesn’t write better as in has new, challenging ideas. Or ways to move/relate to humans on a personal level.

GPT writes clear, concise, authoritative, boring, and generic text.

[simondotau]

To be fair, I rarely write more than a sentence or two in serial form, and I have often determined the point of a sentence before I write the first few words.

Occasionally the act of writing out an idea in words clarifies or changes my mind about that idea, causing me to edit or rewrite what I’ve already written.

[the_other]

You may write one word at a time but the grammar of most languages forces their users to know what they’re going to write several words ahead of the current word.

[FrustratedMonky]

Ok. Why do you think GPT isn't doing that? or is different?

It might be calculating the next word, because you can only write one word at a time, but you can't say that the current next word isn't influenced by a few words ahead. Don't think the current understanding of what LLM's do internally can rule that out.

[JKCalhoun]

And often unsure about where I'm going until I get there, ha ha.

[ofjcihen]

But are you writing to write or are you writing to convey a thought?

[Qwertious]

We ~~all~~ mostly do that partially, but some of us plan our comments ahead and go back and edit previous parts of it after review.