[nirga]

I have it internally, I can share it if you want!

But to the point of comparison between these and tools like Traceloop - it's interesting to see this space and how each platform takes it's own path and finds its own use cases.

LangSmith works well within the LangChain ecosystem together with LangGraph, LangServe. But if you're using LlamaIndex, or even just vanilla OpenAI you'll be spending hours to set up your observability systems.

Braintrust and Humanloop (and to some extend other tools I saw in this area) take the path of "full development platform for LLMs".

We try to look at it as developers look at tools like Sentry. Continue working in your own IDE with your own tools (wanna manage your prompts in a DB or in git? Wanna use LLMs your own way with no frameworks? no problem). We install in your app, with one line and we work around your existing code base and make monitoring, evaluation and tracing work.

[Lienetic]

I'd love to see that list!

[nirga]

Ping me over slack (traceloop.com/slack) or email nir at traceloop dot com