Hacker News new | ask | show | jobs
Writing an LLM Eval with Vercel's AI SDK and Vitest (xata.io)
1 points by gulcin_xata 440 days ago
1 comments

Recently we launched Xata Agent, an open-source AI agent which helps diagnose issues and suggest optimizations for PostgreSQL databases.

To make sure that Xata Agent still works well after modifying a prompt or switching LLM models we decided to test it with an Eval. In this blog, we'll explain how we used Vercel's AI SDK and Vitest to build an Eval in TypeScript.