[mumblemumble]

So, disclaimer: I'm about to focus on the finger instead of where it's pointing. Because that's what I feel like jabbering about right now. TFA isn't just about databases; it makes some interesting points on test practice in general, and is well worth a read.

Anyway,

A story I've seen all too often when mocking the database: A large development effort goes into creating test infrastructure. And then there end up being scads of bugs that weren't caught by the unit tests, because the test doubles for the database don't accurately emulate important behaviors and semantics of the real database.

This isn't just a problem with mocking, mind - it's also a problem I've seen (albeit less often) when using some other DBMS during testing because it's designed to operate in-memory.

Nowadays it's not too hard to configure a RAM disk for the DBMS to use. Especially if your test stack runs it in Docker. If you're having performance problems with your test suite, start there. You might never achieve the same run times as you could with mocking, but, if there's one thing they hammered on in my Six Sigma training that I wholeheartedly agree with, it's that you shouldn't sacrifice quality or correctness for the sake of speed.

It's also not too difficult (not any more difficult than going hog-wild with mocks, anyway) to set up a mechanism that uses transactions or cleanup scripts or similar to ensure test isolation, so I don't find that complaint to be particularly compelling. You can even parallelize your tests if you can set things up so that each test is isolated to its own database or schema.

[UK-Al05]

A good compromise is you write an in-memory version of the dependency and the real version.

You write a comprehensive test of integration tests against the real object.

You then set it up so those same tests can run both the in-memory and real version.

Any differences should show up for the interactions you have specified, and tests should fail.

You can write sociable unit tests against the in-memory version, knowing it matches the same behaviour as closely as you have specified in the tests.

[mumblemumble]

That's what I typically do when I must. I find it yields much more readable tests than mocking does, and they're less likely to accidentally become tautological.

That said, discrepancies can still sink in, so I think it's best to also have some baseline level of tests that run against the real dependency, even if they're typically only run overnight on the CI server.

[dtech]

There's nothing wrong with mocking the DB in unit tests, the kind of tests you're describing are at least integration tests.

[commandlinefan]

> scads of bugs that weren't caught by the unit tests

Which is fine! Unit tests will never catch all the bugs - neither will type safety. Neither will code reviews. Neither will manual testing. But unit tests do catch the kinds of bugs that unit tests are good at catching, which eases the burden on the manual testers.

[monkpit]

> because the test doubles for the database don't accurately emulate important behaviors and semantics of the real database.

Commenter implies that bugs occurred in code that was assumed to be tested and correct (according to the test specs) because it did have tests. Which is decidedly Not Fine.

Now, would I consider them to be “unit tests” in this case? Probably not. But the label you decide to slap on the test doesn’t change the fact that a spec was written and code was tested against it and passed (falsely) due to mocking the db.

[mumblemumble]

Whether, in practice, I would consider them to be unit tests probably depends on what my colleagues want to call them, and little else.

Personally, I do prefer a more classicist definition, because I believe that's the more pragmatic one. But I also believe that arguing over the definition of the term "Unit test" is one of the most wasteful possible examples of bikeshedding. Like you imply, the only thing that really matters is the extent to which your test suite gives you confidence that the software behaves correctly.