[boplicity]

Reading this, I realize that the "prompt" is what's lacking, in terms of the output produced. A writer has a prompt too -- which is created through the entirety of their experience leading up to the moment they sit down to write, often heavily weighted towards the hours leading up to that moment. AI will catch up to actual human writers when it can spend a day's visual, auditory, and linguistic stimuli crafting the prompt that leads to moment of creation.

[inciampati]

Beautifully said. I can add only that the right pattern of prompts may drive the creation of legitimately useful and even beautiful work. Humans need to guide the machines, maybe precisely because they lack the evolutionary drive that gets us up in the morning and keeps us cleaning our babies' behinds.

[m3kw9]

More like a constant chain of prompts, except the prompts are exponentially more data and each output is combined with more sensory inputs for another inference and they happen in real time.

[garba_dlm]

it's not a prompt, it's a dialog, it's many prompts.

[FrustratedMonky]

Yes When I have a conversation with GPT4, it keeps the history, and the entire history is a prompt for the next prompt.

I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.

[inciampati]

But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.

[seanthemon]

Or it can be said that we boil down our experience to a few millions of tokens, only a set of experiences, emotions and often wrong/made up memories.

Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.

[FrustratedMonky]

Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go.

I'd think something like this would be added to the models eventually. Of course, over-simplifying.

[garba_dlm]

I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately

this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations.

I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.

[jprete]

This is rather pseudoscientific.

[FrustratedMonky]

99% of conversations on Internet are pseudoscientific.

Think here we are just 'conceptualizing', 'spit-balling', 'blue-sky-discussion', 'wondering out loud', 'hypothesizing'. etc...