[zingar]

E2E tests in a high ratio to other tests will cause problems. They’re slow and brittle and become a job all on their own. It’s possible that they might help at the start of debugging, but try to isolate the bugs to smaller units of code (or interactions between small pieces of code).

[MoreQARespect]

Hermetic e2e tests (i.e. ones that can run offline and fake apis/databases) dont have that problem so much.

They also have the advantage that you can A) refactor pretty much everything underneath them without breaking the test, B) test realistically (an underrated quality) and C) write tests which more closely match requirements rather than implementation.

[sjdjsin]

> i.e. ones that can run offline and fake apis/databases

I can see a place for this, but these are no longer e2e tests. I guess that’s what “hermetic” means? If so it’s almost sinister to still call these e2e tests. They’re just frontend tests.

> A) refactor pretty much everything underneath them without breaking the test

This should always be true of any type of tests unless it’s behavior you want to keep from breaking.

> B) test realistically (an underrated quality)

Removing major integration points from a test is anything but realistic. You can do this, but don’t pretend you’re getting the same quality as a colloquial e2e tests.

> C) write tests which more closely match requirements rather than implementation

If you’re ever testing implementation you’re doing it wrong. Tests should let you know when a requirement of your app breaks. This is why unit tests are often kinda harmful. They test contracts that might not exist.

[westurner]

> try to isolate the bugs to smaller units of code (or interactions between small pieces of code).

This is why unit tests before e2e tests.

It's higher risk to build on components without unit tests test coverage, even if the paltry smoke/e2e tests say it's fine per the customer's input examples.

Is it better to fuzz low-level components or high-level user-facing interfaces first?

IIUC in relation to Formal Methods, tests and test coverage are not sufficient but are advisable.

Competency Story: The customer and product owner can write BDD tests in order to validate the app against the requirements

Prompt: Write playwright tests for #token_reference, that run a named factored-out login sequence, and then test as human user would that: when you click on Home that it navigates to / (given browser MCP and recently the Gemini 2.5 Computer Operator model)

[skydhash]

And I would add that e2e tests should be more about the businesses rules. Making sure everything is there for a specific flow and not caring that much about the intricacy of things. And such, it should really be part of Ops, not Dev.

Quick feedback with unit tests can help. It can be a pain to decouple stuff so you can test them better, but it’s worth it IMO.