Hacker News new | ask | show | jobs
by dannyw 23 days ago
Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.

Pro is quite limited on the web UI I reckon. This approach can be highly effective for reasonably verifiable task, for example, write comprehensive unit tests pointing out a tricky bug, get multiple agents to swarm at it.
2 comments
nl 23 days ago
It's been very successful at frontier math tasks - a bunch of the Erdos questions have been solved by it - more than any other model.

https://www.erdosproblems.com/

link
cubefox 23 days ago
> Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.

It's unclear how they would do this when there is no signal that provides an objective ground truth.

link