[jdlshore]

This. So much this.

I got involved with Extreme Programming in 2000. Loved it. Best thing since sliced bread, yadda yadda. I was completely spoiled for other kinds of work.

So when that contract ended, I went looking for other opportunities to do XP. But guess what? In 2001, there weren't any. So I started teaching people how to do it. Bam! I'm a consultant.

Several lean years later (I don't mean Lean, I mean ramen), I'm figuring out this consulting thing. I've got a network, I've got a business entity, people actually call me, and oh, oh, and I make a real damn difference.

Then Agile starts getting really popular. Certification starts picking up. Scrum's the new hotness, XP's too "unrealistic." I start noticing some of my friends in the biz are dropping out, going back to start companies or lead teams or something real. But I stick with it. I'm thinking, "Sure, there's some bottom feeders creeping in, but Agile's still based on a core of people who really care about doing good work. Besides, if we all leave, what will keep Agile on track?"

It gets worse. Now I'm noticing that there are certain clients that simply won't be successful. I can tell in a phone screen. And it's not Scrum's fault, or certification, or anything. It's the clients. They want easy. I start getting picky, turning them down, refusing to do lucrative but ineffective short-term training.

Beck writes XP Explained 2nd edition. People talk about Agile "crossing the chasm." I start working on the 2nd edition XP Pocket Guide with chromatic and it turns into The Art of Agile Development. We try to write it for the early majority—the pragmatics, not the innovators and early adopters that were originally attracted to Agile and are now moving on to other things. It's a big success, still is.

It gets worse. The slapdash implementations of Agile now outnumber the good ones by a huge margin. You can find two-day Scrum training everywhere. Everybody wants to get in on the certification money train. Why? Clients won't send people to anything else. The remaining idealists are either fleeing, founding new brands ("Software Craftsmanship"), or becoming Certified Scrum Trainers.

I write "The Decline and Fall of Agile" [1]. Martin Fowler writes "Flaccid Scrum" [2]. I write "Stumbling through Mediocrity" [3]. At conferences, we early adopters console each other by saying, "The name 'Agile' will go away, but that's just because practices like TDD will just be 'the way you do software.'" I start looking very seriously for other opportunities.

That was six years ago.

...

Believe it or not, things haven't really gotten worse since then. Actually, they've gotten a bit better. See, 2-5 years is about how long a not-really-Agile Agile team can survive before before things shudder to a complete halt. But not-quite-Agile was Actually. So. Much. Better. (I know! Who could believe it?) than what these terribly dysfunctional organizations were doing before that they're interested in making Agile work. So they're finally investing in learning how to do Agile well. Those shallow training sessions and certifications I decried? They opened the door.

And so here we are, 2014. I see these "Agile is dying" threads as a good thing. Because they mean that the word is getting out about Agile-in-name-only. Because every time this comes up, you have a horde of people saying "Yeah! Agile sucks!" But... BUT... there's also a few people who say, "No, you don't understand, I've seen Agile work, and it was glorious." That's amazing. Truly. I've come to believe that no movement survives contact with the masses. After 20 years, to still have people who get it? Who are benefiting? Whose lives are being changed?

That means we have a shot.

And as for me... I found that opportunity, so I get to be even more picky about where I consult. But I continue to fight the good fight. Diana Larsen and I have produced Agile Fluency [4], a way of understanding and talking about the investments needed. We've released it, permissive license, for everyone to use. Use it.

Because Agile has no definition, just a manifesto. It is what the community says it is. It always has been. Speak up.

[1] http://www.jamesshore.com/Blog/The-Decline-and-Fall-of-Agile...

[2] http://martinfowler.com/bliki/FlaccidScrum.html

[3] http://www.jamesshore.com/Blog/Stumbling-Through-Mediocrity....

[4] http://www.agilefluency.com

[Tyrannosaurs]

"Believe it or not, things haven't really gotten worse since then. Actually, they've gotten a bit better. See, 2-5 years is about how long a not-really-Agile Agile team can survive before before things shudder to a complete halt. But not-quite-Agile was Actually. So. Much. Better. (I know! Who could believe it?) than what these terribly dysfunctional organizations were doing before that they're interested in making Agile work."

This.

Speaking as a manager struggling to get agile implemented properly I think this is key. Pseudo agile isn't great but it is way better than the pseudo waterfall (or just hacking - in the pejorative sense) that was in place before.

Organisational change is really hard. When the consequences of that change go against everything that's happened before and when they reach a long way from the place trying to make that change (including outside the organsiation to customers in many cases), it's even harder, but it is slowly happening.

[mikevm]

So for someone that wants to study the non-watered down management version of "Agile", which books would you recommend?

And just out of curiosity, are there any popular Agile books that you would not recommend?

[jdlshore]

I'm fond of, um, mine. :-D http://www.jamesshore.com/Agile-Book/

It has a section in the front that describes which pieces to read depending on which role you're in.

[joshyeager]

I second this. My team is about 18 months into our transition to agile, and it is a tough road. This book has been an excellent resource.

The one gap is that it mostly focuses on greenfield projects. It does a good job of highlighting the parts where legacy systems will make agile more difficult, and I understand that spending more time on that would have made the book harder to understand. But we've had to do a lot of learning on our own to try to fill in those gaps for ourselves. With that said, this book does the best job of addressing legacy projects of any that I've read.

If anyone has any recommendations for resources on using agile in legacy systems, I'd love to hear them.

[joshyeager]

Reply to bcobb: I agree that Working Effectively with Legacy Code is great. It's been as revolutionary for our team as The Art of Agile Development.

The big piece that we're missing is how to apply agile practices consistently when our velocity varies so significantly because of legacy code. Feathers' book gave us a long-term path to getting out of that issue, but we're struggling with the planning and communication side of things in the meantime.

[bcobb]

M. Feathers's book "Working Effectively With Legacy Code" is excellent. The focus is on programming techniques to decalcify a legacy codebase, but learning those techniques makes it a whole lot easier to reason about and prioritize work on such a system.

[jdlshore]

I'll repeat bcobb's recommendation for Working Effectively with Legacy Code. It's the bible.

In my Let's Code JavaScript screencast, I've recently started a three-part special on legacy JavaScript code. It follows my efforts to "do it live" with actual legacy code someone else wrote. You might find it useful. (Full disclosure: subscription required, and parts II and III don't come out until April 4th and May 2nd.) http://www.letscodejavascript.com/v3/episodes/lab/6

Edit: I just realized that you already said you're using Feathers' book, and that the challenge isn't technical, it's planning and communication. Let me try again:

Part of that is just "the way it is." At the risk of sounding snarky, I imagine you could plan fairly effectively ("we have huge error bars because our velocity is unpredictable, so given backlog X, we're confident we'll be done sometime within 3-12 months"). See the "Risk Management" and "Slack" sections of AoAD for that.

I also imagine you could communicate that fairly easily ("given our three month estimate, we think we have a 50/50 shot of being done in six months and we're almost certain of being done in less than 12").

For many teams I work with, the problem isn't planning or communication, the problem is that reality isn't conforming to stakeholders' wishes. And there's no way to make it conform. (Short of kicking the problem down the road by accumulating more debt while you search for a new job.)

So how do you break the news?

One way is to, um, not. At this point, stakeholders are used to delays and bugs. The status quo ("we'll pretend to follow your schedule, and you'll pretend to believe us") can be comforting. It might be useful to stick with that.

Another way is to ramp up the frequency of delivery so much that the error bars are less noticeable. An estimate to deliver in 3-12 business days is less painful than an estimate of 3-12 months.

A third option is to negotiate for scope rather than schedule. "We'll absolutely deliver in three months. It might be tiny. That's because of technical debt." (I'm less fond of this approach than I used to be. Negotiating scope makes stakeholders profoundly uncomfortable in a way that slipping dates doesn't, for some reason. Perhaps because they see it as an attempt to weasel out of producing anything.)

Another approach is to stop estimating entirely. "We're working on Feature X and we'll tell you when it's done."

There's also simply explaining the situation: the cynical approach ("we have a ton of technical debt because of the decisions your predecessor made, isn't it great you're here now") or the enlightened approach ("we've got a technical debt problem, it has effects W, T, and F; we're addressing it with P, D, and Q; you can expect consequences O, M, G; we're preventing it from happening again with techniques S, L, A, C, and K.").

I'll typically use some combination of the above depending on the audience.

(I'm in a weird mood. Sorry if this came across as uncaring. The ideas are serious even if the tone is not.)

[joshyeager]

Thanks for the detailed reply! I understood your intent, don't worry about the tone.

We're a small enough team that we all understand the business drivers. The main goal of our agile transition is to improve predictability and clarity. Month-wide error bars on a three-month release don't help those goals much.

Right now we're working to shorten our release cycle and learn how to communicate clearly even when it hurts, on the assumption that shining light on the difficulties will help us find and solve the problems as quickly as possible. But it's definitely a painful transition.

One more point: I agree that negotiating scope with a fixed schedule is difficult. We have done that occasionally. But sales and business people are definitely more willing to accept schedule slippage than scope reduction.

[wpietri]

To James's suggestion, I'd add Beck's Extreme Programming Explained. At this point the book is pretty old, and we've learned a lot since then about how to do things well. But as far as seeing the roots of it, I think that's still a good resource.

[wpietri]

That's really heartening to hear. I checked out 5 or 6 years ago for exactly those reasons. I'm still skeptical that good-Agile can win out over inertia and the very comfortable living bad-Agile consultants and bad-Agile executives are making. But I truly hope that you're right!

[jdlshore]

Me too. :-)