[leobg]

It seems to me that the fastest way for iterative improvement is to use LLMs with pure Python with as few frameworks in between as possible:

You know exactly what goes into the prompt, how it’s parsed, what params are used or when they are changed. You can abstract away as much or as little of it as you like. Your API is going to change only when you make it so. And everything you learn about patterns in the process will be applicable to Python in general - not just one framework that may be replaced two months from now.

[asif_]

I agree that you want to have the flexibility for what you use to build your LLM application. For example, if you want direct API level access to building and searching through your RAG layer, calling your LLM models, and other business logic, you should. There's a lot of opportunities to fine-tune each of these layers. However, you are still left with having thousands of combinations that you can experiment with, ex. which prompt template x rag context x LLM model gives me the best results. And you need a framework that helps you manage these thousands of experiments. That is where I'm trying to position this framework, which is it helps you scale the need for being able to try thousands of different configuration of your LLM application, so you can improve your LLM application performance, while providing as much flexibility for what components you use to actually build your LLM application.

With our framework, if you want flexibility for

> You know exactly what goes into the prompt, how it’s parsed, what params are used or when they are changed

We provide this for you. We just give you a process that lets you try and evaluate different configurations of your LLM application layer at scale.

[TNWin]

I agree. The complexity doesn’t lie in the abstractions that these frameworks are selling.

[asif_]

I try to think about what should be a framework and what should be a library. Libraries are tools that helps you achieve a task, for example, building a prompt, calling LLM models, communicating with vector database.

Frameworks are more process driven for achieving a complex task. This is like ReactJS with their component mode -- they set a process for building web application such that you can build more complex applications. At the same time, you have lots of flexibilities in the implementation details of your application. Framework should provide as much flexibilities as possible.

Similarly, we are trying to build our framework for streamlining the process for LLM development such that you can iterate on your LLM application faster. To help setup this process, we enforce very high-level interfaces for how you build(input & output schema), evaluate, deploy your application. We provide all the flexibilities to the developer for low-level implementation details and ensure it's extensible so you can also use any external tools you want within the framework.