[hpaavola]

All these late "agile has failed" posts are out of touch with reality. They assume that it is always possible to get the best developers and best managers to work with the problem on hand.

I'm a consultant. Officially my title is Senior Test Engineer and usually my job is to go to a company where shit has already hit the fan or will hit in a short time.

The reality is that the company has a bunch of developers who are mediocre at best (and maybe one or two good ones) and managers who do not understand the situation. And of course the budget is already been eaten.

They want to ship their product and make some money with that so they can continue to live their life. If I go there and say hey, you need better developers, more rapid prototyping and managers who are technical enough, nothing will change. Nothing will change because there aren't "rock star" developers avaialble, there isn't time to find new managers and definetly no motivation to change everything right now.

Sure in some sense I might be correct to say so, but my goal is not to show my superiority, but to improve the situation.

But if I teach them a little Scrum, help them setup some CI and so on, they almost always perform better.

And then this statement "Because creating good software is so much about technical decisions and so little about management process, I believe that there is very little place for non-technical managers in any software development organisation."

No. Good software comes from understanding the needs of your customers and meetings those needs. Shitty developers have and will create awesome products just because they know what the users want and need. "Agile" helps even the shitties companies to meet the needs of their customers.

[mattmanser]

I've read your comments a couple of times and still can't really see what you've added.

You're talking about very abnormal situations. Companies that are so technically incompetent they have to hire you.

And then saying that we should listen because in your abnormal experience of dealing with teams so dysfunctional they can't even ship bad software that agile at least gets them going a little bit.

I suspect almost any change would get them going a little bit as the change itself is the trigger.

EDIT: The more I think about it, the more your comment reminds me of the people defending the other fads of years gone past. Workflow diagrams, OOP-design (as a be-all-and-end-all process), UML, Worflow Engines, RAD tools. Your comment reminds me of the defences of them. These things worked because those desperate teams need a light to guide them, not because of the tool itself.

And with each of those tools, it turned out the tool itself is mainly counter-productive to functional teams. I think we're realising Agile is another of those false prophets.

[hpaavola]

The thing is that following situation is really common:

Company has an average team, maybe even bit worse + 1 good person. The managers are not that great either. Everything is average.

The team has been doing something, nobody really knows what, nobody really knows at what pace. Nobody knows what they should be doing.

Occasionally the company releases something which sells a bit and customers give some feedback. Mostly not that great.

When you tech them a bit of Scrum, help them get builds every night and help managers to try out the product once in a while (remeber, not all software is a website, some are airport weather observation systems). And help the company to get some of their customers to try out the software once in a while even though it is not "ready".

Suddenly the results of the team is visible to interest groups and interest groups needs are visible to developers, managers can get some idea of the velocity and everybody can have a better idea where we are, where we should go and when we might get there. Then iterate and improve.

What we have done here is that we improved the quality of the company with really little cost with our existing talent pool.

Yes, this is really common.

[ahomescu1]

> Company has an average team, maybe even bit worse + 1 good person. The managers are not that great either. Everything is average.

I think the application of the OP here is "fire the bad/mediocre managers, put that 1 good tech person in charge". Couldn't they do that?

[exelius]

I disagree. The "companies so technically incompetent they have to hire consultants" are the norm. Think Fortune 500 companies developing internal software tools here; a lot of these guys are still stuck in the "we release software every year and hope the requirements haven't changed" mindset. Meanwhile the "customer" expectations have changed: people expect faster releases because Google, Facebook, Netflix, etc. update their shit every month. The business environment changes quickly too; the old development ways just aren't fast enough to keep up.

Agile helps force groups like this, who have been doing things a certain way for a long time, to re-think how they manage requirements and release schedules. Most likely, the developers at the bottom level have been screaming about this for years, but middle management doesn't hear them because they're stuck in meetings all day having to explain why all their projects are over budget because change management costs are through the roof. Along comes Agile as a solution to all of this.

This isn't a technical problem, it's a management problem. So yeah, a lot of these companies hire consultants to come in and tell them how/what to do because consultants have no skin in the game and aren't involved in the typical petty management squabbles.

[ctb_mg]

I also disagree with the premise that these situations are very abnormal. Ours is a big field which expands outside of sunny California, where you can't walk a few feet without bumping into a rockstar coder. In the rest of the world where the mediocre is the status quo, Agile gets team members organized and motivated. Of course, if you could find rockstars, you'd hire them, solve many problems, and ship your product. Not everyone has the pleasure of that kind of engineer pool.

[mattmanser]

I actually live in Nottingham, UK.

Here there's dev meetups, Second Wednesday, Notts Tuesday and probably more regular events I don't even know about.

I've talked and worked with a lot of developers round here now as well as interviewed with a fair few companies and done their technical tests and talked to their lead developers.

A lot of them write better code than I've seen in the github repos of Californian 'rockstars'. And yes, there's plenty of bad code too, out-dated practices, people asking me if I know VB.Net. And yes one of the companies I worked at (oh so briefly) had terrible code.

No different from the Californian mecca I imagine. Do you think that only good developers go to SV. There's no magic forcefield keeping bad developers out of California. In fact because SV is so desperate you might even argue they probably have a lower average standard.

There's plenty of shipping, successful products coming out of this city and successful IT Teams working for other industries. And bad ones.

[twic]

The question is whether this is abnormal or not. I suspect that it is not. That is, i suspect that there are many more mediocre-to-poor developers and managers out there than there are good-to-excellent ones.

[hpaavola]

Exactly. Then the questions is how we can make most of us to perform better. The answer is not to "be excellent developer". The answer must be in the way we do things.

[_ea1k]

I disagree with your assertion that this is a "very abnormal situation". However, I think that I agree with your main assertion.

The improvement isn't really specific to bringing in Agile, but rather to giving the team a clear focus, and way to measure their improvement in deliverables. There are many ways to do that, it's just that Scrum happens to be a relatively straightforward and lightweight way to introduce that in a way that is compatible with many of these teams.

Unlike a lot of those bad approaches, though, I don't think Scrum is necessarily a hindrance to a properly functioning team. There are a lot of good ideas in Scrum that are designed to get obstacles out of the way, IMO.

[chiph]

It's not that they're incompetent - it's that they don't think about their work abstractly. They're intently focused on cranking out code, and anyone who pops his head up and says "Wait a second, is this the right thing to do?" gets taken aside by management and told: "I need you to focus on meeting your deadlines, not that other stuff".

Agile has to start with management adoption. And not Fake-Agile™ either. I've seen a PowerPoint slide a few times now that lists "Iteration 0 - Envisioning" as a bullet point. It's like RUP got painted with Agile terminology. And everyone in the room nods their heads knowingly because it's what they've been doing all along, only now they're Agile.

[joesmo]

The situation described above is far and above the norm in the industry. Bad code, bad management, bad practices, bad developers are all the norm. The abnormal situation is one where things aren't FUBAR. FUBAR is the norm.

[collyw]

"No. Good software comes from understanding the needs of your customers and meetings those needs. Shitty developers have and will create awesome products just because they know what the users want and need."

Depends what you class as "good software". My colleague left some code which "worked" and "met the needs" (at the time). I made a small change to the database it interacted with and it fell over. I had to go in a debug it, but it was so badly written I had to completely refactor it just to debug it properly.

So her software met the needs, my rewrite of it met the needs, and has been far more reliable and maintainable. I would say that is better software.

[hpaavola]

Of course technically better implementation is better if the only thing that changes is the technical quality of the implementation.

But it does not follow that doing good software is a technical challenge. It does not matter how fast your algorithm is if nobody wants the results that it spits out.

Defining good software is really hard task, but in general what I mean by it is that good software increases the well-being of its users. Yes, one aspect of that is the technical quality of the implementation.

[usea]

What happens when the needs of the customers or users change? What happens when your understanding of their needs was imperfect and needs to be updated? Is the software that met the old needs still good? Was it ever good?

What happens when the software could only ever meet the old needs, but is impossible to update to meet the new needs? Was it ever good?

Needs change. The ability for software to be altered to meet evolving needs is not simply a "technical quality of the implementation." It doesn't just have a faster algorithm. It's better software.

[hpaavola]

I'm not sure what part we disagree here. Yes, technical quality matters. But it matters way less than delivering something that users need. And yes, better technical quality ceteris paribus equals better software.

[usea]

We disagree whether meeting the needs of your users is a technical challenge. The ability for software to meet changing needs is a technical quality of the software.

[hpaavola]

The ability for software to meet the needs is mostly a technical problem. After all, it is a feature of the implementation. But that is not the bigger problem. The bigger problem is to figure out the needs and that is not a tecnical problem.

[raverbashing]

Good software is the one that fulfills a need and pays the bills, period.

The Moon Lander software on the Apollo spacecraft probably had several defects, was unmaintainable, but it got the job done several times.

MS-DOS had several defects, was more maintainable than the above software sure, but it transformed MS.

Sure, it is better when the software is maintainable, more flexible, etc, and I don't advocate against it. What I do advocate against is overthinking all of this and not shipping.

[enraged_camel]

It was your fault. You made a modification to a production database without understanding the full repercussions. You brag about how your refactoring made her software better, but you are ignoring the fact that you got into that mess because of a failure to follow best practices and common sense.

[collyw]

Sure it was my fault she used functions within functions because she didn't know how to understand Pythons module system. It was my fault she had big nested hashes without any comments to explain the structure. It was my fault she had two variables "Indexes" and "indexes" in the same function. I must be the shitty coder, not her....

I at least try and make my code maintainable.

[enraged_camel]

She may be a shitty coder [1]. But her code worked at the time she wrote it, right? And it only broke after you went in and fiddled with the database without first checking to see what systems and programs were using that database and how your changes would affect those systems and programs, right?

[1]Or she may not be. Often times when you develop a system or even write a piece of code, you operate under certain constraints that might make it impossible to make it fool-proof. You may also not have the opportunity to come back and refactor it later. Maybe that was the case when she wrote her code. You may have had the time to debug what she wrote and even re-write it from scratch. She may not have had that luxury.

Heck, maybe she wasn't even a professional programmer. Maybe she had just taught herself some basic programming and wanted to apply it to the job to add value to the company and succeeded in doing so, and she had no idea that some douchebag would come in at a later date, break her program and then blame her for being a shitty programmer. The point is: her code worked at the time she wrote it. It was you who broke it. End of story.

[watwut]

What best practices do you imagine him not to follow?

[efsavage]

Agree. "Agile" methologies are a great way for:

1. Good managers to manage new or mediocre engineers. 2. Good engineers to control new or mediocre managers.

No system will completely mitigate bad managers or bad engineers, and if they're both good the right system will manifest itself.

[briantakita]

Agile is the new RUP.

The main audience is shifting toward the shitties. It's the circle of life for cultural movements. Nothing wrong with Agile. It's just not my thing anymore.

[at-fates-hands]

I've actually come to the conclusion Agile propagates poor developers and poor managers.

I've known some really good developers who go belly up in Agile systems because they can't cope with the tight deadlines, having to ship something in two weeks, or people constantly breathing down their necks.

Over the years, I feel like Agile has been used to deliver mediocre products, and in turn, turned good developers into shitty developers simply because the "business" wants to ship something a lot sooner than it should be. It's a downward spiral since all they care about is something getting built, not the inherit quality of said product.

Hence, its because of Agile there are shitty developers and shitty products being shipped, not the other way around.

[hpaavola]

By definition most companies, most teams and most individuals are average. There are very few really good developers and managers. What suits those few does not really matter. They will do great things not matter what. Agile works for most better than the alternatives.

[Tyrannosaurs]

Unless you have a very oddly skewed set of people, by the definition of average most people are average (or close to it)...

[vanderZwan]

I think you missed the part where he implied that what programming culture considers "good" software is nothing short of a near-perfect to perfect technical implementation. In other words: what the other guy called "shitty" is actually the average.

See also the odd thing that most people consider themselves better than average, which by definition cannot be true.

[Tyrannosaurs]

Yes, the idea that almost everyone considers themselves to be a better than average driver.

I've always wondered with that whether the problem is with the question. What it's really checking is whether someone will admit to being worse than average, which is a very different thing to thinking you're worse than average.

[ZeroGravitas]

In "thinking fast and slow", I'm sure the author gave an example of something people will all admit to being below average at.

Googling it, the example given is "starting conversations with strangers" and his theory is that people ignore the question they are asked ("how they compare with average") because it's hard/impossible to actually answer, and instead, without realising it, substitute an easy question ("are you good at x") and answer that instead.

[briantakita]

I'd like the flip that thinking on it's head a bit. It's not about "average" or "better or worse" than average. The need to put yourself and others into some sort of ranking stems from the need to categorize everything into a hierarchy.

I'm talking about something more binary. Good or bad. Progress or stagnation. The right or wrong side of history.

I'm also talking about power. Who determines engineering practice. The engineers or the marketers?

Evidence of power is in the corporate hierarchy. Are there n tiers of middle management? Can an engineer truly make a difference at a company? Will the engineer be able to have a voice in what the priority is?

[vanderZwan]

If you want to reduce a problem of this complexity to binary choices, be my guest, but I doubt it will get you anywhere useful.

[briantakita]

I agree.

As an engineer who aspires to be on the leading edge of the practice, I've become more picky with who I work with. Luckily, "those few" tend to also be picky and seek each other out.

When I first started out, I sought people who were entrepreneurial and innovative.

[Tloewald]

If you get people who aren't customer focused, aren't testing, and are wasting huge amounts of time in pointless meetings focus on their customers, do continuous builds, tests, iterative releases, and have fewer meetings I suspect you can and will reap significant benefits.

But that isn't "agile" it's just some good practices with a label du jour. Good software companies were doing nightly builds twenty years ago (when doing such builds was a major technical accomplishment). The agile cargo cult against which the writer is ranting is far larger, more insidious, and counterproductive than that. E.g. Our project is infested with "release train engineers", "coaches", "rally consultants" and other highly paid morons who actively damage the project while claiming credit for any improvements we manage to accomplish behind their backs.

[kasey_junk]

I don't disagree about the hypocrisy of the folks who have redefined the Agile movement to be something that is decidedly not Agile.

That said, the thing I dislike about this new "Agile has failed" meme is that it discounts how much the movement has changed the status quo. Sure it make sense now that small iterative releases, continuous integration, automated tests, customer collaboration, etc will produce better software. And yes there were teams that did this before Agile became popular, but it certainly wasn't the norm.

There used to be no standard way to write/run unit tests. The class of software called continuous integration server did not exist. Gant charts were completely normal planning devises.

I've worked in environments that literally believed that a senior developer could use uml to design the entire system down to the method level and then it would be a trivial matter for a junior developer to "fill it in". That sounds insane now, but it happened throughout the software industry.

Agile didn't fail, it worked so well it became the norm, and now the term is being co-opted by opportunists. That's sad, but let's not let that detract from the very real success story of the Agile software movement.

[Tloewald]

I agree with much of what you've said, but I have to pick on this remark:

"And yes there were teams that did this before Agile became popular, but it certainly wasn't the norm."

And there are now lots of teams not doing it but calling what they do "agile" because they're engaging in cargo cult practices. A whole bunch of stupidity is simply painted with agile terminology (e.g. unfocused daily meetings that are referred to as "standups").

I'm sure "healthcare.gov" was "agile".

[wpietri]

I think you've mainly missed the point of his article, but let me address the last bit, about non-technical managers.

The way to reconcile his view and yours is to have parallel engineering and product management structures. There are non-technical product managers, but there are no non-technical engineering managers. Every team has engineers and a product manager, but neither reports to the other.

That way you get people dedicated to understanding needs without putting them in charge of things they aren't qualified to understand.

[exelius]

If you have two parallel organizations that interact tightly but have no power over each other, you end up needing a mediator. This usually ends up being project management, or "the process". This is where you get the mixed bag in companies with mediocre talent levels -- usually they don't have anyone who understands the needs of both the engineers and the business. The level of process usually either chokes the engineers' productivity or ends up not giving the business what they need.

Again, it's possible to do, but it takes talent that understand both the business needs and how to manage a development organization (note: this is not the same as knowing how to code.) That's rare to find through a corporate HR process.

[bunderbunder]

I work in an team that's organized the way the parent describes, and there's no mediator needed. The engineering managers and product managers have a good working relationship and collaborate closely, so they usually have no problem reaching consensuses that work for everyone. The close collaboration means that each side is aware of (and can usually anticipate) the other side's needs and limitations, but the division of responsibilities ensures that the number of balls getting dropped is minimized.

The relationship most certainly does not need a mediator. Sticking a mediator in there would probably end up destroying the arrangement. If the two camps can't work together without one, a better solution is to figure out who it is that doesn't know how to play well with others and get them replaced.

[exelius]

I'm not saying it can't work that way -- just that it often doesn't if you don't have good people. Also, even if you have good people, when the product group is more or less a single silo and you have a dozen different engineering teams working on various modules of a product, communications get lost and balls get dropped and you need some sort of communication process that ensures at a bare minimum accountability (which usually ends up at Agile).

I know a lot of people would say "Well just hire better people," but that's often not realistic. It takes time to hire people, corporate HR processes are terrible at differentiating good developers from mediocre ones. In the time it takes you to hire people, you still need to ship product.

[bunderbunder]

Hum. Well I would say that if the product group is a monolithic silo interacting with a dozen engineering teams, that does sound like a recipe for trouble. The system I'm thinking of involves having at least one member of the product team for every engineering team.

That actually edges back toward my one serious complaint about Agile development: On the teams I've worked with, there's a bit too much value placed on generalism over specialism in team members' skill sets. Anything that's everybody's responsibility is effectively nobody's responsibility.

[exelius]

Yeah; a lot of the solutions you see pushed here on HN don't really scale past a startup-sized company. A lot of times if you have a very large product, you have multiple engineering teams working on a layered architecture. For example, the product guys will hand off a requirement as a user story that describes "As a customer, I can push the green button and it orders a pizza sent directly to my desk." That user story involves the front-end team (UI mockups, interface implementation, etc.), the back-end team (now has to support storing and retrieving the user's pizza preferences), the billing systems team (now has to ensure the pizza billing process has the proper audit checks, etc. on it), the deployment team (who now has to ensure pizza-order messages are allowed through the firewall to the pizzeria), etc.

It's a bit of a contrived example, but you get the idea. Unless you have a centralized management function, all of this doesn't happen and your glorious pizza button either doesn't work or fails deployment. Usually the onus falls on the product manager to do this, but with so many teams involved it can be a nightmare.

[wpietri]

I have seen a lot of companies work fine this way, so I disagree.