[TOGoS]

> There’s another important difference too. In an ordinary Notebook there’s a “temporal thread of evaluation” in which inputs and outputs appear in the sequence they’re generated in time

I've always found this kinda confusing, and the "evaluate everything above" approach used by the LLM cells much more intuitive, but having the two types of cells behave so differently brings it back to confusing and asking for trouble. Do you think it would make sense to have an option to force all cells to work that way?

[theodoregray]

Hoo boy that's a long and complicated question. I think there might actually be an option you can set on cell groups, or perhaps at the notebook level, to force re-evaluation of everything above every time you evaluate a cell (which would be the closest one could do to implementing the same behavior for kernel evaluations as for chat evaluations).

Why do I, who would have been the one to implement such a feature, not even remember whether it exists or not? Because if it does, it's horribly useless. It's just rarely what you actually want, and if you do want to do that, you can simply select the cell group, or command-A select the whole notebook, and use shift-return to evaluate everything. That's something I do regularly, but only when I need to, and it would be horrible if it tried to do that every time. I think it was Maple that tried such an interface. (Except their's was 2-dimensional: you could move equations/definitions around on the screen, and anything below or to the right would use definitions from above or to the left of it. This interface was one of the things that convinced me that this was a bad idea.)

The situation with GPTs is different: they do not have the equivalent of kernel state. It took me a while to believe it's actually true, but they have infinite amnesia: every time you send an API call, or submit a new prompt in their web interface, the model is starting completely from scratch with no state or memory whatsoever of anything that's happened before.

That fact that it looks like you're having an interactive conversation in which it knowns what you said before is an illusion created by the fact that the entire conversation all the way back to the start is re-submitted with every new prompt.

Given that reality, and the way people seem to want to interact with these things, the use of a physical context rather than a temporal one seemed most natural. We did talk about implementing our own temporal context mechanism to artificially make the LLM appear to have time-based internal state, like a kernel, but I personally hated the idea, and we never implemented it.

Now, it's possible that at some point these systems will in fact start having internal state (i.e. you have your own personal LLM running on a wad of gum stuck behind your ear, and as you interact with it it constantly fine tunes its weights, or something). At that point we might regret the decision, but I feel like this whole world is so unstable and rapidly evolving that it made sense for us to do what's right now, and worry about the future, if there is one, during that future.

Theodore