[weisser]

Congrats on the launch!

I messed around with the playground onboarding...here's the output:

With Memory Mem0.ai I know that you like to collect records from New Orleans artists, and you enjoy running.

Relevancy: 9/10

Without Memory I don’t have any personal information about you. I don’t have the ability to know or remember individual users. My main function is to provide information and answer questions to the best of my knowledge and training. How can I assist you today?

Relevancy: 4/10

--

It's interesting that "With Memory" is 9/10 Relevancy even though it is 100% duplication of what I had said. It feels like that would be 10/10.

It's also interesting that "Without Memory" is 4/10 — it seems to be closer to 0/10?

Curious how you thinking about calculating relevancy.

[soulofmischief]

This is why in my system I have more specific, falsifiable metrics: freshness, confidence, etc. which come together to create a fitness score at the surface-level, while still exposing individual metrics in the API.