[cletus]

My second project at Google basically killed mocking for me and I've basically never done it since. Two things happened.

The first was that I worked on a rewrite of something (using GWT no less; it was more than a decade ago) and they decided to have a lot of test coverage and test requirements. That's fine but they way it was mandated and implemented, everybody just testing their service and DIed a bunch of mocks in.

The results were entirely predictable. The entire system was incredibly brittle and a service that existed for only 8 weeks behaved like legacy code. You could spend half a day fixing mocks in tests for a 30 minute change just because you switched backend services, changed the order of calls or just ended up calling a given service more times than expected. It was horrible and a complete waste of time.

Even the DI aspect of this was horrible because everything used Guice andd there wer emodules that installed modules that installed modules and modifying those to return mocks in a test environment was a massive effort that typically resulted in having a different environment (and injector) for test code vs production code so what are you actually testing?

The second was that about this time the Java engineers at the company went on a massive boondoggle to decide on whether to use (and mandate) EasyMock vs Mockito. This was additionally a waste of time. Regardless of the relative merits of either, there's really not that much difference. At no point is it worth completely changing your mocking framework in existing code. Who knows how many engineering man-yars were wasted on this.

Mocking encourages bad habits and a false sense of security. The solution is to have dummy versions of services and interfaces that have minimal correct behavior. So you might have a dummy Identity service that does simple lookups on an ID for permissions or metadata. If that's not what you're testing and you just need it to run a test, doing that with a mock is just wrong on so many levels.

I've basically never used mocks since, so much so that I find anyone who is strongly in favor of mocks or has strong opinions on mocking frameworks to be a huge red flag.

[throwaway7783]

I'm not sure I understand. "The solution is to have dummy versions of services and interfaces that have minimal correct behavior".

That's mocks in a nutshell. What other way would you use mocks?

[cletus]

Imagine you're testing a service to creates, queries and deletes users. A fake version of that service might just be a wrapper on a HashMap keyed by ID. It might have several fields like some personal info, a hashed password, an email address, whether you're verified and so on.

Imagine one of your tests is if the user deletes their account. What pattern of calls should it make? You don't really care other than the record being deleted (or marked as deleted, depending on retention policy) after you're done.

In the mock world you might mock out calls like deleteUserByID and make suer it's called.

In the fake world, you simply check that the user record is deleted (or marked as such) after the test. You don't really care about what sequence of calls made that happen.

That may sound trivial but it gets less trivial the more complex your example is. Imagine instead you want to clear out all users who are marked for deletion. If you think about the SQL for that you might do a DELETE ... WHERE call so your API call might look like that. But if the logic is more complicated? Where if there's a change where EU and NA users have different retention periods or logging requirements so they're suddenly handled differently?

In a mokcing world you would have to change all your expected mocks. In fact, implementing this change might require fixing a ton of tests you don't care about at all and aren't really being broken by the change regardless.

In a fake world, you're testing what the data looks like after you're done, not the specific steps it took to get there.

Now those are pretty simple examples because there's not much to do the arguments used and no return values to speak of. Your code might branch differently based on those values, which then changes what calls to expects and with what values.

You're testing implementation details in a really time-consuming yet brittle way.

[throwaway7783]

I am unsure I follow this. I'm generally mocking the things that are dependencies for the thing I'm really testing.

If the dependencies are proper interfaces, I don't care if it's a fake or a mock, as long as the interface is called with the correct parameters. Precisely because I don't want to test the implementation details. The assumption (correctly so) is that the interface provides a contract I can rely on.

In you example, the brittleness simply moves from mocks to data setup for the fake.

[MrJohz]

The point is that you probably don't care that much how exactly the dependency is called, as long as it is called in such a way that it does the action you want and returns the results you're interested in. The test shouldn't be "which methods of the dependency does this function call?" but rather "does this function produce the right results, assuming the dependency works as expected?".

This is most obvious with complex interfaces where there are multiple ways to call the dependency that do the same thing. For example if my dependency was an SQL library, I could call it with a string such as `SELECT name, id FROM ...`, or `SELECT id, name FROM ...`. For the dependency itself, these two strings are essentially equivalent. They'll return results in a different order, but as long as the calling code parses those results in the right order, it doesn't matter which option I go for, at least as far as my tests are concerned.

So if I write a test that checks that the dependency was carried with `SELECT name, id FROM ...`, and later I decide that the code looks cleaner the other way around, then my test will break, even though the code still works. This is a bad test - tests should only fail if there is a bug and the code is not working as expected.

In practice, you probably aren't mocking SQL calls directly, but a lot of complex dependencies have this feature where there are multiple ways to skin a cat, but you're only interested in whether the cat got skinned. I had this most recently using websockets in Node - there are different ways of checking, say, the state of the socket, and you don't want to write tests that depend on a specific method because you might later choose a different method that is completely equivalent, and you don't want your tests to start failing because of that.

[3uler]

The fakes vs mocks distinction here feels like a terminology debate masking violent agreement. What you’re describing as a “fake” is just a well-designed mock. The problem isn’t mocks as a concept, it’s mocking at the wrong layer. The rule: mock what you own, at the boundaries you control. The chaos you describe comes from mocking infrastructure directly. Verifying “deleteUserById was called exactly once with these params” is testing implementation, not behavior. Your HashMap-backed fake tests the right thing: is the user gone after the operation? Who cares how. The issue is finding the correct layers to validate behavior, not the implementation detail of mocks or fakes… that’s like complaining a hammer smashed a hole in the wall.

[throwaway7783]

In the SQL example, unless you actually use an SQL service as a fake, you cannot really quite get the fake do the right thing either. At which point, it's no longer a mock/fake test but an integration/DB test. Network servers are another such class and for most parts can be either mocked or faked using interface methods.

I would argue that (barring SQL), if there are too many ways to skin a cat, it is a design smell. Interfaces are contracts. Even for SQL, I almost end up using a repository method (findByXxx flavors) so it is very narrow in scope.

[Jach]

The general term I prefer is test double. See https://martinfowler.com/bliki/TestDouble.html for how one might distinguish dummies, fakes, stubs, spies, and mocks.

Of course getting overly pedantic leads to its own issues, much like the distinctions between types of tests.

At my last Java job I used to commonly say things like "mocks are a smell", and avoided Mockito like GP, though it was occasionally useful. PowerMock was also sometimes used because it lets you get into the innards of anything without changing any code, but much more rarely. Ideally you don't need a test double at all.

[phanimahesh]

There are different kinds of mocks.

Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by.

The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build.

Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.

[OrangeMusic]

The second way is usually referring to as "fakes", which are not a type of mocks but a (better) alternative to mocks.

[bonesss]

In reflection heavy environments and with injection and reflection heavy frameworks the distinction is a bit more obvious and relevant (.Net, Java). In some cases the mock configuration blossoms to essentially parallel implementations, leading to the brittleness discussed earlier in the thread.

Technically creating a shim or stub object is mocking, but “faking” isn’t using a mocking framework to track incoming calls or internal behaviours. Done properly, IMO, you’re using inheritance and the opportunity through the TDD process to polish & refine the inheritance story and internal interface of key subsystems. Much like TDD helps design interfaces by giving you earlier external interface consumers, you also get early inheritors if you are, say, creating test services with fixed output.

In ideal implementations those stub or “fake” services answer the “given…” part of user stories leaving minimalistic focused tests. Delivering hardcoded dictionaries of test data built with appropriate helpers is minimal and easy to keep up to date, without undue extra work, and doing that kind of stub work often identifies early re-use needs/benefits in the code-base. The exact features needed to evolve the system as unexpected change requests roll in are there already, as QA/end-users are the systems second rodeo, not first.

The mocking antipatterns cluster around ORM misuse and tend to leak implementation details (leading to those brittle tests), and is often co-morbid with anemic domains and other cargo cult cruft. Needing intense mocking utility and frameworks on a system you own is a smell.

For corner cases and exhaustiveness I prefer to be able to do meaningful integration tests in memory as far as possible too (in conjunction with more comprehensive tests). Faster feedback means faster work.

[throwaway7783]

Why is check if XYZ is called with return value ABC bad, as long as XYZ is an interface method?

Why is a minimally correct fake any better than a mock in this context?

Mocks are not really about order of calls unless you are talking about different return values on different invocations. A fake simply moves the cheese to setting up data correctly, as your tests and logic change.

Not a huge difference either way.

[rcxdude]

The point is to test against a model of the dependency, not just the expected behavour of the code under test. If you just write a mock that exactly corresponds to the test that you're running, you're not testing the interface with the underlying system, you're just running the (probably already perfectly understandable) unit through a rote set of steps, and that's both harder to maintain and less useful than testing against a model of the underlying system.

(And IMO this should only be done for heavyweight or difficult to precisely control components of the system where necessary to improve test runtime or expand the range of testable conditions. Always prefer testing as close to the real system as reasonably practical)

[throwaway7783]

But mocks are a model of the dependency. I don't quite see how a fake is a better model than a mock.

In any case, I agree testing close to a real system, with actual dependencies where possible is better. But that's not done with a fake.

[rcxdude]

The kind of mocks the OP is arguing against are not really a model of the dependency, they're just a model of a particular execution sequence in the test, because the mock is just following a script. Nothing in it ensures that the sequence is even consistent with any given understanding of how the dependency works, and it will almost certainly need updating when the code under test is refactored.

[ahepp]

Mocking is testing how an interface is used, rather than testing an implementation. That's why it requires some kind of library support. Otherwise you'd just on the hook for providing your own simple implementations of your dependencies.

[yearolinuxdsktp]

Heavy mocks usage comes from dogmatically following the flawed “most tests should be unit tests” prescription of the “testing pyramid,” as well as a strict adherence to not testing more than one class at a time. This necessitates heavy mocking, which is fragile, terrible to refactor, leads to lots of low-value tests. Sadly, AI these days will generate tons of those unit tests in the hands of those who don’t know better. All in all leading to the same false sense of security and killing development speed.

[fatso83]

I get what you are saying, but you can have your cake and eat it too. Fast, comprehensive tests that cover most of your codebase. Test through the domain, employ Fakes at the boundaries.

https://asgaut.com/use-of-fakes-for-domain-driven-design-and...

[LgWoodenBadger]

“The solution is to have dummy versions of services and interfaces that have minimal correct behavior”

If you aren’t doing this with mocks then you’re doing mocks wrong.

[bccdee]

Martin Fowler draws a useful distinction between mocks, fakes, and stubs¹. Fakes contain some amount of internal logic, e.g. a remote key-value store can be faked with a hashmap. Stubs are a bit dumber—they have no internal logic & just return pre-defined values. Mocks, though, are rigged to assert that certain calls were made with certain parameters. You write something like `myMock.Expect("sum").Args(1, 2).Returns(3)`, and then when you call `myMock.AssertExpectations()`, the test fails unless you called `myMock.sum(1, 2)` somewhere.

People often use the word "mock" to describe all of these things interchangeably², and mocking frameworks can be useful for writing stubs or fakes. However, I think it's important to distinguish between them, because tests that use mocks (as distinct from stubs and fakes) are tightly coupled to implementation, which makes them very fragile. Stubs are fine, and fakes are fine when stubs aren't enough, but mocks are just a bad idea.

[1]: https://martinfowler.com/articles/mocksArentStubs.html

[2]: The generic term Fowler prefers is "test double."

[cowsandmilk]

In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production.

Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.

[sfn42]

I use test containers and similar methods to test against a "real" db, but I also use mocks. For example to mock the response of a third party api, can't very well spin that up in a test container. Nother example is simply time stamps. Can't really test time related stuff without mocking a timestamp provider.

It is a hassle a lot of the time, but I see it as a necessary evil.

[supriyo-biswas]

You can use a library like [1] to mock out a real HTTP server with responses.

[1] https://www.mock-server.com/

[pdpi]

I'd go a bit farther — "mock" is basically the name for those dummy versions.

That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.

[rgoulter]

> there is a massive difference between writing mocks and using a mocking library like Mockito

How to reconcile the differences in this discussion?

The comment at the root of the thread said "my experience with mocks is they were over-specified and lead to fragile services, even for fresh codebases. Using a 'fake' version of the service is better". The reply then said "if mocking doesn't provide a fake, it's not 'mocking'".

I'm wary of blanket sentiments like "if you ended up with a bad result, you weren't mocking". -- Is it the case that libraries like mockito are mostly used badly, but that correct use of them provides a good way of implementing robust 'fake services'?

[pdpi]

In my opinion, we do mocking the exact opposite of how we should be doing it — Mocks shouldn't be written by the person writing tests, but rather by the people who implemented the service being mocked. It's exceedingly rare to see this pattern in the wild (and, frustratingly, I can't think of an example off the top of my head), but I know Ive had good experiences with cases of package `foo` offering a `foo-testing` package that offers mocks. Turns out that mocks are a lot more robust when they're built on top of the same internals as the production version, and doing it that way also obviates much of the need for general-purpose mocking libraries.

[saghm]

I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.

[jchw]

The difference, IMO, between a mock and a proper "test" implementation is that traditionally a mock only exists to test interface boundaries, and the "implementation" is meant to be as much of a noop as possible. That's why the default behavior of almost any "automock" is to implement an interface by doing nothing and returning nothing (or perhaps default-initialized values) and provide tools for just tacking assertions onto it. If it was a proper implementation that just happened to be in-memory, it wouldn't really be a "mock", in my opinion.

For example, let's say you want to test that some handler is properly adding data to a cache. IMO the traditional mock approach that is supported by mocking libraries is to go take your RedisCache implementation and create a dummy that does nothing, then add assertions that say, the `set` method gets called with some set of arguments. You can add return values to the mock too, but I think this is mainly meant to be in service of just making the code run and not actually implementing anything.

Meanwhile, you could always make a minimal "test" implementation (I think these are sometimes called "fakes", traditionally, though I think this nomenclature is even more confusing) of your Cache interface that actually does behave like an in-memory cache, then your test could assert as to its contents. Doing this doesn't require a "mocking" library, and in this case, what you're making is not really a "mock" - it is, in fact, a full implementation of the interface, that you could use outside of tests (e.g. in a development server.) I think this can be a pretty good middle ground in some scenarios, especially since it plays along well with in-process tools like fake clocks/timers in languages like Go and JavaScript.

Despite the pitfalls, I mostly prefer to just use the actual implementations where possible, and for this I like testcontainers. Most webserver projects I write/work on naturally require a container runtime for development for other reasons, and testcontainers is glue that can use that existing container runtime setup (be it Docker or Podman) to pretty rapidly bootstrap test or dev service dependencies on-demand. With a little bit of manual effort, you can make it so that your normal test runner (e.g. `go test ./...`) can run tests normally, and automatically skip anything that requires a real service dependency in the event that there is no Docker socket available. (Though obviously, in a real setup, you'd also want a way to force the tests to be enabled, so that you can hopefully avoid an oopsie where CI isn't actually running your tests due to a regression.)

[zem]

my time at google likewise led me to the conclusion that fakes were better than mocks in pretty much every case (though I was working in c++ and python, not java).

edit: of course google was an unusual case because you had access to all the source code. I daresay there are cases where only a mock will work because you can't satisfy type signatures with a fake.

[ebiester]

Mockito, in every case I had to use it, was a last resort because a third party library didnt lend itself to mocking, or you were bringing legacy code under test and using it long enough to refactor it out.

It should never be the first tool. But when you need it, it’s very useful.

[rurban]

I dont use dummy services and I dont use mocking. I'm writing simulators to test things for HW or big services which are not available for testing.

Simulators need to be complete for their use cases or they cannot be used for testing.