[talos_]

There's a LOT of LLM frameworks out there. The "bring your own framework" adds flexibility instead of endless integrations (like in ML libraries and MLOps tooling).

If I'm getting started with LLMs or have 1-2 POC deployed at my company what's the benefit of adding burr to my stack?

[elijahbenizzy]

So I'm a big fan of limiting complexity -- you generally don't want to add something to your stack unless you get value from it.

If you're getting started with LLMs, the first step is an API call to OpenAI (or some other foundational model) to get a feel for it, then think about how you want to integrate with your application. Burr can help standardize the structure, allowing you to think less about it at any given point (as logic is encapsulated into actions). Furthermore, the UI can help you debug.

With a few POCs at your company (assuming you want to iterate and get into production), Burr can help abstract some complex parts away (state management and telemetry), and make your code cleaner and more extensible, which can help you iterate quickly and explain what you're doing (you can always draw a picture of your app).

We think that it buys companies something they really want -- the ability to swap out implementations/vendors as it decouples the logic from application concerns.

So, if you have logic that's starting to get complicated (and might get more-so), I think Burr is good. If all you're doing is a single wrapper over a GPT-call, it might be overkill (or not! things tend to grow in complexity).

[talos_]

That makes sense. I'll look into it more!

One project was to create a layer between external APIs (OpenAI, Gemini, Claude) and our team. This way we can manage the cost of API calls, try different providers, and log API usage to find out what works / what doesn't.

How does burr scale with an application and support multiple users?

[elijahbenizzy]

Cool! Yeah I think that would pair nicely. You can have the abstraction at the action level, or push it down into the stack (as it seems you're doing). Have also seen some startups building proxy layers for OpenAI/GPT.

It's just a library so it depends on what could be the bottlenecks. Standard ones are: 1. Compute -- just scale horizontally 2. DB -- scale horizontally (and vertically), use intelligent partitioning

(1) is generally easy with autoscaling, (2) is where your queries/data modeling matter, and this is why we made persistence entirely pluggable, figuring we wouldn't know the best usage patterns yet. More reading: https://burr.dagworks.io/concepts/state-persistence/.

One more thing to think about -- Burr isn't opinionated about how you build your app -- E.G. up to you to think about what happens if you have your webapp open in, say, two tabs. You'll need to manage situations like this, but Burr's plugin capabilities should make that easier to iterate on. We're looking to add more examples on this -- also reach out if you have any specific questions/find anything missing/have suggestions.