[muzani]

Yup, I meant "Markov chain" as a way to say state. The idea was that it was extremely complex to control state. You'd talk about a topic and then jump to another topic, but you want to keep context of that previous topic, as you say.

Was RAG popular on release? Google Trends indicates it started appearing around April 2023.

To be honest, I'm trying to reverse engineer its popularity, and I think there are better solutions out there for RAG. But I believe people were already using Langchain as GPT 3.5 was taking off, so it's likely they changed the marketing to cover RAG.

[authorfly]

I don't think this is a sensible use of Markov chain because that has historic connotations in NLP for text prediction models and would not include external resources in that.

RAG has been popular for years including in models like BERT and T5 which can also make use of contextual content (either in the prompt, or through biasing output logits which GPT also supports). You can see the earliest formal work that gained traction (mostly in 2021 and 2022 by citation count) here - http://proceedings.mlr.press/v119/guu20a/guu20a.pdf - though in my group, we already had something similar in 2019 too.

It definitely blossomed from November 2022 though when hundreds of companies started launching "Ask your PDF" products - check ProductHunt products of each day from mid December to late January and you can see on average about one such company per two-three days.

[muzani]

Gotcha. I started using langchain from two angles. One was dumping a PDF with customer service data on it. Nobody called it RAG at the time but it was. It was okay but didn't seem that accurate, so I forgot about it.

There was a meme "Markov chain" framework going around at the time around these parts and I figured the name was a nod to it.

It was to solve the AI Dungeon problem: You lived in a village. The prince was captured by a dragon in the cave. You go to the blacksmith to get a sword. But now the village, cave, dragon, prince no longer exist. Context was tiny and expensive, so the idea was to chain locations like village - blacksmith - cave, and then link dragon to cave, prince to dragon, so the context only unfolds when relevant.

This really sucked to do with JS and Promises, but Langchain made it manageable. Today, we'd probably do RAG for that in some form, it just wasn't apparent to us coming from AI Dungeon.