Hacker News new | ask | show | jobs
by MoreQARespect 247 days ago
Hermetic e2e tests (i.e. ones that can run offline and fake apis/databases) dont have that problem so much.

They also have the advantage that you can A) refactor pretty much everything underneath them without breaking the test, B) test realistically (an underrated quality) and C) write tests which more closely match requirements rather than implementation.
1 comments
sjdjsin 247 days ago
> i.e. ones that can run offline and fake apis/databases

I can see a place for this, but these are no longer e2e tests. I guess that’s what “hermetic” means? If so it’s almost sinister to still call these e2e tests. They’re just frontend tests.

> A) refactor pretty much everything underneath them without breaking the test

This should always be true of any type of tests unless it’s behavior you want to keep from breaking.

> B) test realistically (an underrated quality)

Removing major integration points from a test is anything but realistic. You can do this, but don’t pretend you’re getting the same quality as a colloquial e2e tests.

> C) write tests which more closely match requirements rather than implementation

If you’re ever testing implementation you’re doing it wrong. Tests should let you know when a requirement of your app breaks. This is why unit tests are often kinda harmful. They test contracts that might not exist.

link
westurner 247 days ago
> try to isolate the bugs to smaller units of code (or interactions between small pieces of code).

This is why unit tests before e2e tests.

It's higher risk to build on components without unit tests test coverage, even if the paltry smoke/e2e tests say it's fine per the customer's input examples.

Is it better to fuzz low-level components or high-level user-facing interfaces first?

IIUC in relation to Formal Methods, tests and test coverage are not sufficient but are advisable.

Competency Story: The customer and product owner can write BDD tests in order to validate the app against the requirements

Prompt: Write playwright tests for #token_reference, that run a named factored-out login sequence, and then test as human user would that: when you click on Home that it navigates to / (given browser MCP and recently the Gemini 2.5 Computer Operator model)

link