|
For non-urgent workflows I mostly use Batch APIs. As you said, the bare batch APIs are a pain to use. On top of that, I would add that most async-to-batch libraries force users to learn a new framework or refactor their existing code, which is a huge friction in itself. I've been in those trenches as a developer and I decided to create a literal 2-liner python lib that gets you from async to batch: https://github.com/vienneraphael/batchling You don't need to change your code and it supports most providers (Anthropic, Gemini, Groq, Mistral, OpenAI, Together, Vertex, XAI, Doubleword) and all imagineable python frameworks (Langchain, PydanticAI, Instructor, DSPy, LiteLLM, Pydantic Evals, ..) |