[CharlesW]

> I think a framework like this should provide abstractions that allow people to focus on the important part like prompt engineering and productionizing the app and worry less about how to figure out stuff like this.

I appreciate the detailed explanation, thank you!

If you'll permit another noob question, can/should frameworks like this include source attribution for responses? Based on articles like https://jamesg.blog/2023/04/02/llm-prompts-source-attributio... I'd guess the strict answer would be no, and that any strategies you might use to elicit them might hallucinate citations.

[spstoyanov]

This is a very hard problem. If you base your response on a text that has been returned from a vector DB you can reference this text. But if you rely on a citation generated directly from the LLM is probably impossible to reliably guarantee the source is real. I guess if it’s a link you can check if it’s actually a real link, scrape the content and do some extra trickery to figure out if it contains the text produced by the LLM but I’ve never tried this myself.

[CharlesW]

> If you base your response on a text that has been returned from a vector DB you can reference this text.

Oh, sweet! That would be a wonderful feature to have in a framework. For personal or corporate knowledge bases, sources would be really helpful for people who want to dig deeper.