[jondwillis]

I’m preparing for bringing a generative Agent app into production. Currently I am using langsmith for observability and shopping around for benchmarks and automated test suites. To be honest, the automated evaluation feature is PART of my product. So, i’m wondering if you have an API or SDK for interacting with the evaluator (such that I could show results in my own UI)

[virtuallyvivek]

Gentrace co-founder here.

That's an interesting use. We don't have that currently, since we haven't had anyone request workflows only using our evaluators as a service.

That said, we would like every part of our application to be controllable through API requests eventually, which would naturally support your use case.

Would you be open to diving deeper into your requirements? The best way to reach us is at [e-mail]