[politelemon]

Similar tool I was about to look at: https://github.com/promptfoo/promptfoo

I've seen this in both tools but I wasn't able to understand: In the screenshot with feedback, I see thumbs up and thumbs down options. Where do those values go, what's the purpose? Does it get preserved across runs? It's just not clicking in my head.

[krawfy]

For now, we just aggregate those across the models / prompts / templates you're evaluating so that you can get an aggregate score. You can export to CSV, JSON, MongoDB, or Markdown files, and we're working on more persistence features so that you can get a history of which models / prompts / templates you gave the best scores to, and keep track of your manual evaluations over time.