[visarga]

There is an alternative explanation - both AI and humans piggyback on language. Language patterns encode and express all the emotions, and they have been created by evolution. We are, much like LLMs, contextual language generators

Language is our repository for both intelligence and emotion, it has its own evolution and replicates faster than biology. I don't pin AI abilities to the models, but to the datasets they are trained on, knowledge created by our own hard work and risk taking over millennia

Admitting the essential role of the language corpus in AI over models would change discussion about the speed of AI evolution and its risks. Language is not something we can control, it is emergent from the whole population. But at the same time it looks unlikely to have an exponential growth as knowledge comes with hard work and risks. Iterating in our imagination doesn't produce new knowledge, it is all crystallised feedback and experience from the world. It's also how the scientific method works.

[atodorov99]

Humans don't piggyback on language (assuming you mean) for intelligence and emotions. Emotions predate language by miles and they don't even have to be consciously thought of, much less expressed through language. Doubly so for people who do not have an inner voice and do not even use a language to think. Body language exists as the simplest example of this.

As for intelligence I honestly believe the mind uses whatever tools it sees fit for a task. Sometimes when you think about a problem it is through words, sometimes it is through visuals and sometimes it is just felt.

[kristiandupont]

I had a similar realization recently. I plugged my blog post(1) two days ago and I am doing it again now so I will try and refrain for a while, but it really sticks with me: after trying to create a chat bot for a while, I feel like I have gained significant insights into how humans work. It all seems so simple now! It's a weird and awesome feeling!

1) [link redacted]

[visarga]

> To address this, I implemented a strategy of tagging messages to create and utilize categories.

I think before RAG we need to do more legwork with the LLM on the raw text. Here is one of my blog posts that is related:

https://mindmachina.wixsite.com/ai-blog/post/the-promise-of-...

The idea is to create chain-of-thought annotations from your raw texts, that would improve the embedding and retrieval process by making implicit things explicit.

For example "the last letter of this message" would not embed similar to "e", but if it was annotated with CoT, it would work.

[kristiandupont]

Interesting, thank you!

I think a lot about the cost of the loop, mostly in terms of time. I don't want the bot to take too long to respond. That's why dream cycles seem like an obvious solution to some of the more heavy work. I guess it would make sense to combine those with your idea -- "given what I know about the user, what should I study?", especially if it has access to an "enhanced" knowledge db like you suggest..

[visarga]

Yes, it would be a good idea for an agent first to collect user interests, and later, when ingesting data in the RAG system, to annotate it with useful metadata such as topic, summary, entities, user interest related question-answer pairs. Whatever we want to ask later better be made explicit in the text.