[mikepurvis]

Is there anyone exploring or writing about this in public? I've felt for a while that the turn-based model was not quite right, but also felt too stupid and ill-informed to have much of an opinion about what else it could be.

[b112]

I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next.

But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look at the sentence and determine if it should interrupt and start a real turn with thinking on. Which kind of is non-turn based in a way. If you're typing fast, it might hit that run every 3 or 4 words, but that's sort of how a human might be when a person is talking to them. That is, waiting for enough info to interrupt, if needed.

There might be a way to process chunks of a sentence using commas as break points, eg for comma delimitated phrases in sentences, so the whole sentence doesn't need to be re-processed each "should I break in" assessment at word break.

Could be fascinating. Could actually do some of this right now.

I don't think this is what the parent poster was thinking, but the idea even at this level seems fun.

[mikepurvis]

Yeah, I've played with some similar stuff on my 9070xt. But ultimately all the ceremony on top is cloaking that it's still just two or more models taking turns prompting each other to give the illusion of continuous thought. It's still one thought at a time, with every thought starting from scratch with a big chunk of prior context.

The idea of true continuous thought and memory-generation is very interesting, though I can't even begin to conceive of how it would work.

Or if it's even correct? Maybe our brains are secretly actually turn based too?

[b112]

I think they're definitely attention based. They're just immensely faster than LLMs, because a lot of processing is in silicon in a sense. Think of a ball flying towards you, you don't have to think, the data is handed to your conscious mind, speed, direction, which literally knows how to snag the ball out of the air.

But we have multiple things vying for attention, and some are immediate. Being on the phone talking to someone with great attention, and then touching a burning surface -- you immediately pull your hand back (lizard brain) before even being aware you're doing it. The same with peripheral vision and something surprising coming at you from the side. It snags your attention.

So maybe we are turn-ish based, but just multiple parallel processes each with their own turn? Neurons have their own 'trigger', and I think the brain has layers of triggers, each aggregating and filtering up to the top which then triggers.

I think doing this all with an LLM is silly, some of it should be innate, such as peripheral vision. Data handed to the main thread when triggers occur. I wouldn't want an LLM to handle "walking" fully either.

Some octupus have a sub-brain in each tentacle, each thinking and feeling, there are serious questions as to what its mind is like. I feel initial LLM powered androids may have to be like this a bit.

[KoolKat23]

I agree with you however I think even then you're still giving our brains too much credit. The speed definitely comes from that processing being "in silicon".

Your ball throwing example however will be handled by really small and really fast "fine tuned agents" dedicated to catching that ball. Eyes to motor neuron system. There are the illusion of free will experiments that demonstrate your brain only rationalises and explains whatever activity took place after the fact (It's explanation may even be entirely wrong).

[chorsestudios]

Thinking Machines, the started founded by former OpenAI CTO Mira Murati. The interaction models demo’s in their videos imo breaks the awkward turn-based barrier. Returning responses quickly reaches a threshold where it starts to feel like a natural conversation. Their approach to solving this problem is rather clever.