[mikekchar]

I've tried about 3 times to respond to your comment because I believe it deserves a response. Alas, I am not yet expert enough to write what needs to be said in a format smaller than a book ;-)

Hopefully a few clues will lead you to a better understanding of what I am trying to convey. There is a difference between the artifacts you are building and the practices you are using to build them. When you are pondering different solutions to the user's problems, you are pondering about the artifacts that you wish to build. In such things, you must be as free as you can be.

On the other hand, the practices you use to build those artifacts must necessarily be more strict. If you use a haphazard approach to choosing your practices, you will not be able to refine them. If you choose complicated practices that require large amounts of coordination with other people, you will likewise not be able to refine your practices. The goal, IMHO, is to simplify your practices and maximize your facility with them.

There isn't just one way to write software. You need to choose practices that will work for you and your team. The choice of those practices is of paramount importance to the success of your team. It must be things that you are good at and that you can coordinate efficiently on. There isn't a one size fits all. Having said that, you must be very strict about what you are doing because (at least in my experience) most practices are not compatible. Even very subtle changes or misunderstandings can cause various practices to misalign and backfire.

So far, what I have said is not specific to "agile" processes. To talk about agile in a public forum is difficult because for many of us old guys the term has been diluted considerably. So keep in mind that this is my opinion (a non-famous, random guy on the internet) and there will be luminaries that disagree (possibly violently ;-) ).

I believe that an "agile" process is one in which the main artifact (the software that you are delivering) maintains at least a constant potential for modification over time. Your goal is maximize throughput over time. In my experience, you can actually accelerate throughput over time with the right team/practices. I will steal Ron Jeffries term, "hyper productivity" to refer to this condition.

In more clear terms, what you want to say is that at the beginning of the project adding a feature is easy. If you are "agile" then a month or a year from now it is just as easy. If you are "hyper productive", then a month from now it is easier to add a feature. A year from now it is much easier to add features.

My assertion (which I can not back up in such a small space) is that if you are inconsistent applying your practices, then you will never reach the level I call "agile", much less "hyper productive". Some people might call this kind of inconsistency, "pragmatism". I do not know if you would be one of those people. I will say, though, that one of the reasons I have put such an effort into writing this response is that I recognise your name from other postings you have made and think that it is not a wasted effort ;-)

I have experienced what I refer to as "hyper productivity" on a few projects. It is magical. I know of only one way to get there. In the 30 years I've been in this industry I have never seen a flexible approach to practices achieve it. That doesn't mean it is impossible, but I am doubtful.

[lukesan]

Thank you for taking time and the effort to response. I appreciate that. I agree with the notion that you need to be free in choosing what you want to build. I like the notion of why, what, how in http://abbytheia.com/2016/02/07/why-what-how/

So we should be free in what we build but how should be influenced 1) by the what and 2) your team's experience. The why is key to know what to build. Without knowing why, pondering about what is pointless.

The constant potential for modification in an efficient manner is indeed a key of software development and very difficult to achieve in my experience. Besides concentrating on small things you need to watch and maintain your architecture of the system. Alas there is a point where you need to change the architecture. I think Martin Fowler was it what found out that at a certain point an architecture which was a right fit at the beginning cannot be maintained in a reasonable way and has to be stepped up. I would be interested what are the combination of practices that work for you is (even if they do not apply to my team). With pragmatism I refer to the artifact, the what, it needs to be a good fit to the problem and the users. And it is important to thing with what solution can "you get away with". Not in the way it is build but what does it achieve, the capability it provides. I have seen too many developers failing to fulfill requirements in time and budget because they stuck to the wording of the requirement. The requirement was a box for them, a prison. The problem behind the requirement is the one the developer needs to solve, not the requirement as is which is often wrong or unclear anyway. I have seen so much leverage here that I am putting most of my effort here: teaching to find and eliminate assumptions, getting to the problem, goals and needs of the user and the business, ...

The practices, the how. In my 15 years of professional development I tried many different practices and some are good for some situations, others feel like a waste of time and effort. I am yet on the search for a consistent set that fits me and my team. How do you consistently apply the practices? Only in projects? Or do you use katas or something like that?

Again thank you for discussing. I believe such honest and clear discussions are needed more than the many arguments about what is better or worse.

[mikekchar]

Ha ha! That's a very nice cartoon. I really like it :-)

Picking practices is very hard because it is highly dependent upon the people. Personally, I prefer to use a mostly unmodified set of XP practices. As there are many confusing descriptions of these practices I encourage you to look at this one: http://c2.com/cgi/wiki?ExtremeProgrammingCorePractices

As I said, I don't really modify these practices except for the coding standard. That is to say, I have one, but I actually prefer to appoint an arbiter for coding standards rather than maintain a document. It reduces a potential point of politics where people try to game the coding standards to get their way (yes, unfortunately it happens... sigh).

One of the problems with this is that programming practices are very subtle and are surprisingly interdependent. Kent Beck has talked about trying to create the minimal set of "generative" practices: practices, which when followed, generate the behaviour you want without needing to state it. I think this is a good idea, but it requires that you truly understand how the practices work together.

A good example is testing. Unit testing has gotten reasonably popular. Some people focussed on the word "unit" and decided that they should mock all of the collaborators in their tests. Then they found that their tests were brittle when they refactored. Quite a few people were happy with this and just decided not to refactor very much because, hey, they thought about the design up front a lot and probably got it right. Right at that point, you have broken your XP practices because XP practices require constant incremental design.

Other people will think, "I'll replace these unit tests with integration tests" because they aren't as brittle. But integration tests require a lot of set up with lots of things in motion. So you end up with slow tests. It's fine because, hey, 0.1 seconds is still pretty damn fast for a test, isn't it? But then because you need to find all the corner cases, you write lots and lots of tests and your test suite takes an hour to run. Right there you have destroyed your XP processes because you can't do TDD any more.

Of course, the answer is that "unit testing" was always just a word and you shouldn't "test" units like you would a black box. It goes on and on and on. It's super subtle and really easy to get wrong. However, because the practices are generative, you can actually test that you are doing the right things in the manner I described above. Breaking one of the 12 XP practices will break at least one of the others. It's a matter of keeping you eyes open to the things you are breaking.

Which all reads like an advertisement for XP. Rest assured that my biases are simply formed because this is what I have experience with. I'm sure that other successful shapes exist. Indeed I often work with teams that are not equipped to handle all of XP. In those cases I try to find alternative approaches. I meet with varying degrees of success, but have never even come close to "hyper productive" without full on XP.

You next question is a good one. How do I consistently apply the practices? Personally, I practice a lot (I try to put in at least an hour a day outside of working hours). I do katas and work on side projects. The group I work with is also keen on practice and we have "hack time" and organized katas during work hours. But just like in sports, training is fine but there is no substitute for the real thing.

Usually there is a lot of pressure when you are working. At the end of the day, though, the programmer is programming and has a lot of power to decide how the practices work. The question always come down to: are you willing to do it?

I've worked as an "agile coach" quite a bit (although I always do a lot of programming). One day one of my younger colleagues came up to me and said, "I could fix this quickly or I could fix this right. Which one should I pick?" I answered, "Are you expecting me to suggest you choose the wrong solution?" He happily went back to his task. Of course, my head is now on the block if we get it wrong. :-)

Are you able/willing to make the same kinds of decisions? It is not possible in many situations. This can be fine, but my experience is that one will not be able to build an amazing team if you can not.

One last thing, which may sound patronizing. When I was about 15 years in, it was the time when I first started realizing how everything fit together. It is very encouraging for me to hear that your are on that same search. I think you will find what you are looking for very soon.

[lukesan]

Thanks a lot.