[henrik_w]

I don't agree with "making code easy to test tends to over-abstract it". The dependencies are there no matter what, but they become more obvious when you try to test the code in isolation. I think the main benefit of TDD is the way it forces you to break the program apart in smaller, less dependent pieces. The end result is better structure and less dependencies between parts.

[barrkel]

I fundamentally disagree with you.

If the goal in writing software is to reduce it to a set of pieces that plug together, I'd agree. But it's an arbitrary metric and not an absolute measure of quality, not by a long shot.

Note that I don't say "composable", because composability is something that needs to be designed in, and it isn't usually clear how to do it best until the third or so time around - the rule of threes.

Furthermore, I don't say reusable, because reusability is something that also needs to be designed for. In particular, reusability in such a way that software can evolve over time without breaking clients (reusers) of the abstraction demands a tight and constrained contract that is broadened carefully, while testable components demand broad and flexible contracts, otherwise not everything would be available to be tested.

Every part that testing has forced you to break out to be individually addressable is a part that you cannot remove in a refactoring that significantly changes the way a library solves its problem.

A library that has been broken into parts that are neither composable nor reusable is simply over-complex. Almost every extant Java library is like this!

Of course, if you just write end software in small teams, none of this is relevant to you. But it is crucial in library design, especially when client code is outside of your organization.

[Peaker]

If you split it into little parts, and those parts have no sensible meaning on their own or no sensible interface and semantics -- then you're absolutely right. Testing such parts will be difficult, too, because of this.

But if instead of a monolith, you have a set of components with well defined interfaces that have simple semantics (that do not leak abstractions) -- whether or not these parts are re-usable in other contexts -- then you almost automatically have higher quality software:

* Easier to test means it will likely be better tested

* Well-defined interface and abstraction and a small implementation means that reviewing/correctness becomes easy. You only need to understand a small component to review it. "Obviously no bugs" rather than "no obvious bugs"

* Easier to split the work across developers

* Easier to comprehend the whole as a collection of its parts

The total number of lines of code, or even the total complexity may increase relatively to a monolithic design. But correctness becomes so much easier.

You mention refactoring, too, and IME, refactoring can be both easier or more difficult, depending on whether it is within a single small component or across multiple ones.

If you add architectural/design changes -- then it is night and day. A monolith will likely have to be rewritten to make an architectural change reliably.

A set of components can easily be split, for example, so that a few components are moved to run on a different system with a network protocol between them.

[henrik_w]

A program made up of loosely coupled pieces has several advantages over one that is more monolithic. It is composable – all you have to do is put the parts together, then you have your complete program. It is easier to modify, because parts can be swapped out with minimal impact. It is easier to test, since the parts can be tested in isolation, which makes testing the complete program much easier.

As for reuse, I like this quote: "Don't aim for reuse. Write small, independent components you can reason about, and the right pieces for reuse will fall out." Jessica Kerr @jessitron on Twitter

[taeric]

How did that work out for the Kernel debates? :)

I don't think anyone disagrees that a "well designed and written" program of loosely coupled pieces has advantages over a monolithic one.[1] The debate is really over which is easier to do. And, the argument you are responding to is essentially, that it is easier to abstract out parts after you have done it in whole a few times. I know, personally, that that is a very compelling argument.

[1] Well, there probably is some debate on the feasibility of making things as loosely coupled as you would like. Back to the kernel debate, how many microkernels have survived with the device support of linux?

[spand]

The kernel debate is a different one.

Software can easily consist of loosely coupled pieces as source code and be compiled and run as a single monolith with hardly any performance loss at runtime (versus coupled source code).

[taeric]

Maybe I misrepresent the debate, then. My understanding is that the debate was that there was no future in a monolithically sourced and run kernel. Linus took the position that while that had a certain appeal, he just wanted an operating system he could use. If anybody had managed to deliver on the microkernel dream, he would probably not have started the linux kernel.

That is to say, that there is appeal to the "loosely coupled" dream of a software solution. Not just in source but in execution. However, there is the reality that this is very hard. The contention in this thread is that to think you can start in the loosely coupled set of parts is very ambitious.[1] It isn't that it is a bad goal. Just that it is akin to wanting to score well in a marathon without first running a few smaller races.

[1] Unless I am misrepresenting that, of course.

[jerf]

The real problem with the debate here is that a kernel is a place where performance is in your top two concerns, fighting it out with correctness, and among other things beating out "effort to create" and "skill level needed to create". If your software does not have performance as its absolute #1 criterion, and you care about the effort it takes to create it and the skill level needed to create it, you'll probably want to go back to easily isolated pieces that can be tested and understood without the whole system being understood, and that may not perform the absolute best that they could. (Although I find this software doesn't produce slow software on its own; at most it costs you a few more pointer traversals than you may like. Slow software is IMHO far more likely to come from highly coupled programs that everyone is terrified to optimize lest the whole thing come apart.)

Trying to use the kernel as a template to guide all software development is not a great idea.

[memracom]

The Linux kernel is not monolithic in the sense that people are talking about here. There are modules or parts in the Linux kernel that are composed using carefully crafted APIs. The kernel difference is that the design goals are different from most applications and that Linux leverages every possibly way to integrate software components on a von Neuman architecture. It goes well beyond what you normally do in a business app.

Linux is well designed and you can learn from it, but in order to get value from that study you need to be a skilled C programmer and at the top of your game. Therefore it is a bad example for people who mainly use other languages.

In additon lets not forget that the SOLID principles, DRY, YAGNI and so on, are not hard and fast rules. Every extreme programmer will regularly violate those principles. The purpose of the principles is to guide your work, to make you see clearly what you are doing, so that when you violate a principle you do it for a good reason.

[ollysb]

One of the biggest problems I find with people adopting TDD is that they create one test per class. This was never the intention. The idea was that you test units, which may be a single class but equally may also be an aggregate made up of smaller classes. Tests should pin down external behaviour without overly restricting internals. Mocking at every class boundary leads to brittle tests suites that aren't focused on APIs, not to mention being a PITA to refactor.

[hderms]

I really appreciate your response. It's very measured and, to me, seems the result of experience, as opposed to gung-ho idealism. People need to realize that software is complicated, and that despite the fact that computers operate on unambiguous rules, that the people creating the software often can't depend on concepts in the same way. Following test-driven development fanatically is not a shortcut to designing re-usable, composable software. There is no shortcut to that.