Hacker News new | ask | show | jobs
by what 12 days ago
You can write tests that cover every line/branch without actually verifying the code does what it’s supposed to. Relying on (unreviewed?) LLM written tests that provide coverage may not be a great idea.
1 comments

So, a project can have 100% test coverage (the tests run each line or branch at least once without any errors or failures) but fail acceptance tests.

If there is prompt insufficiency, there is probably acceptance test insufficiency.

A more assuming agent could automatically develop a plan that includes presumptive acceptance tests and request feedback before spending tokens