[desc]

Silver bullets. They don't exist.

Code review. Read the results of someone thinking through a process. Spot more than they will, simply by throwing more eyes at it. Actually fairly effective: getting a senior dev to cast even a lazy eye over everything gives more opportunities to discuss Why It's Done This Way and Why We Don't Do That and Why This Framework Sucks And How To Deal With It with specific concrete examples which the other dev is currently thinking about. But it's still easier to write the code yourself than review it, and things still get missed no matter how careful you try to be, so it's still just another layer.

Unit tests. They cover the stuff we think to check and actually encountered in the past (ie. regressions). Great for testing abstractions, not so great for testing features, since the latter typically rely on vast amounts of apparently-unrelated code.

Integration tests. Better for testing features than specific abstractions, and often the simplest ones will dredge up things when you update a library five years later and some subtle behaviour changed. Slow sanity checks fit here.

UI-first automation (inc. Selenium, etc). Code or no-code, it's glitchy as hell for any codebase not originally designed to support it; tends to get thrown out because tests which cry wolf every other day are worse than useless. Careful application to a few basics can smoke-test situations which otherwise pass internal application sanity checks, and systems built from the start to use it can benefit a lot.

Manual testing. Boring, mechanical, but the test plans require less active fiddling/maintenance because a link changed to a button or something. Best for exploratory find-new-edge-cases, but throwing a bunch of students at a huge test plan can sometimes deliver massive value for money/coffee/ramen. Humans can tell us when the instructions are 'slightly off' and carry on regardless, distinguishing the actual important breakage from a trivial 2px layout adjustment or a CSS classname change.

So that's the linear view. Let's go meta, and combine techniques for mutual reinforcement.

Code review benefits from local relevance and is hampered by action at a distance. Write static analysers which enforce relevant semantics sharing a lexical scope, ie. if two things are supposed to happen together ensure that they happen in the same function (at the same level of abstraction). Encourage relevant details to share not just a file diff, but a chunk. Kill dynamic scoping with fire.

Unit and Integration tests can be generated. Given a set of functions or types, ensure that they all fit some specific pattern. This is more powerful than leveraging the type system to enforce that pattern, because when one example needs to diverge you can just add a (commented) exception to the generative test instead of rearchitecting lots of code, ie. you can easily separate sharing behaviour from sharing code. Write tests which cover code not yet written, and force exceptions to a rule to be explicitly listed.

UI testing is rather hard to amplify because you need to reliably control that UI in abstractable ways, and make it easy to combine those operations. I honestly have no idea how to do this in any sane way for any codebase not constructed to enable it. If you're working on greenfield stuff, congratulations; some of us are working on stuff that's been ported forwards decade by decade... Actual practical solutions welcome!

That's my best shot at a 2D (triangular?) view: automated tests can enforce rules which simplify code review, etc. The goal is always to force errors up the page: find them as early as possible as cheaply as possible and as reliably as possible.

The machine can't check complex things without either missing stuff or crying wolf, but it can rigidly enforce simple rules which let humans spot the outliers more easily.

And it is amazing how reliable a system can become just by killing, mashing and burning all that low-hanging error-fruit.

[ozim]

Code reviews should be about project structure and abstractions and keeping approach in order or to use team common approach instead of each team member doing whatever, well syntax/code should be linted and formatted automatically nothing for reviewer. Second thing is checking by second pair of eyes if they understand code in question in the same way.

Unit and integration tests should not be generated. Those should be written by people if they find code that they are writing doing complex things like some specific calculation. It is more as a tool for understanding what you are doing and then maybe leave some tests behind for regression. But don't generate BS tests that will only slow down system and people. People have to understand what is going on and be on top of it and never "just run the tests" because tests that are passing green but are actually wrong are really bad.

UI testing should not be abstractable - it should be only augmenting manual UI testing - so tester should be automating his own work after he has done it manually. That tester should also find things that take him long time or have to be done multiple times and are not changing often so he wins time to do more important things. QA person should also be always engaged with the system and automation because that is the only way you can keep domain knowledge.

[desc]

It depends how you count the unit/integration tests, really.

If you've got a general rule which must apply across an entire system, generate the necessary tests so that they fail granularly and don't require messing around to find the exact case which breaks. IMO that's one test, just applied to a range of cases.

An example might be mappings for Entity Framework (or similar ORMs, etc). Auto-generated migrations simply do not work if you need to limit migration downtime and maintain certain data invariants (which can't be specified in the schema, and yes, those always exist). So you need to write database migrations manually. This introduces risk of desync of entity mappings and schema.

So don't just spot-test roundtripping entities (a nontrivial system will have hundreds and something always gets missed). Instead, write a tool to introspect the DB schema and the Framework's mappings, and check that they match sufficiently closely. Every time someone adds an entity or property or something, it's already covered.

Similar cases exist when dealing with any interface between separate systems, especially when you don't control one of them. If you're regularly mapping between two models, use something like Automapper which can be asked to verify its mappings to check that every property is handled in some way.

(Granted, Automapper doesn't catch everything, but it builds a model that could probably be introspected over to spot encountered bugs and check that other possible examples of those bugs don't exist. Doing so generatively catches future additions of possible cases for free. If you're really paranoid, define some means of marking manually-written tests which cover each case, and test that a test exists for each case.)

Computers are really good at force-multiplication. They should trivially be capable of spotting other instances of known categories of bug. This is not hard to do and doesn't require wooly nebulous machine-learning shit: we've had introspectable ASTs since the dawn of compilers.

[failwhaleshark]

The only silver bullet is approximated by a holistic approach.