[edulix]

Instead of long learning or long contexts, at some point artificial neural networks will have to transition to continuous/online learning - learn while using the network. This way, limitations are broken like they are in our minds.

Similar to what Numenta HTM networks do, but scalable and performant for real use cases.

BTW, perhaps human-like conscience emerge as a "self-attention-like" mechanism between context and learning. Just saying.

[qumpis]

Learn how? I think having infinite context is perfect - no need to learn on my data online and risk exposing it to others.

[thomasahle]

Alternatively we need the model to have a long term memory, and be able to load stuff to/from that while reading.

[sr-latch]

My hunch is that the next progression in AI will incorporate forward forward training