[lordnacho]

There's really one issue here. Budget.

You want two pairs of eyeballs on everything as you write it? Okay. Pay two guys.

You want complete test coverage? Okay. Can take as long as writing the code itself. Maybe more.

You want full documentation? Okay. Guy can't spend as much time on coding.

You want new APIs to be included? Okay. Guy's got to read docs and try samples. Less time.

What I've seen is you always end up with someone noticing that you could just spend all your time coding, and you would still produce an output that was ok. At least for small projects, or the start of large ones. It's as if there's a true productivity that's much lower than the imagined, and whoever is setting priorities doesn't notice it. He only sees code that's been written. He doesn't see technical debt. He doesn't see when a test catches a budding error. He doesn't see what that second pair of eyes is for. He doesn't know why you're always reading about some new thing.

[omouse]

The issue is underestimation. Everything is underestimated. That bug that was 5 lines of code? You had to spend 3 days looking for it.

Writing docs and unit tests now is cheaper than fixing bugs later.

I guess you're right that it is about budget because we're all trying to build a spaceship when we're only given some duct tape and a shoebox.

I've been explicitly told in one project to avoid quality and just make something cheap and fast. The project has been in that mode for the last year. Funny how cheap and fast actually means more expensive and slower.

On another project, the manager created different columns in our Kanban board suggesting that we must write unit tests and must have code reviewed and QA'd. Okay Mr Manager, now tell me where in your schedule and budget you're going to fit those key activities when you've accepted a fixed-time, fixed-cost project.

Underestimation is rampant in the industry and I really like this saying in order to justify the necessary work, "If you have time to do it over, you have time to do it right."

[ploxiln]

The funny part of this, is that often the "budget" of time / effort is low-balled, and you end up with something that's more expensive to fix. The Hubble telescope from the essay is a great example of this. And we've all seen the project where the decision was to just put in a hacky work around for a mis-design because the project deadline is in less than a month... and it continues that way for over a year.

In these cases, some sort of unwillingness or inability to pause and be thoughtful, driven by a rush to meet a budget or deadline, cost a lot of money and time. Perhaps, as you suggest, it's the great difficulty of the lay-person or the manager judging the quality of software (or engineering I guess), and naturally selecting for the "cheapest" quality that seems to work, and very often hitting "too cheap".

[libria]

The part about delivering a large visible project on time and owning the CEO or sales team's promises I actually get. Sometimes there's just better market/brand value in delivering on time (even if it's crap underneath).

What is truly criminal is not giving the developers at least a couple refactor iterations to fix all the monkey patch bandaiding put into making that happen. It's a mix of both management's naivete about how bad the underlying code is and their greed of wanting to cash in on the present victory at the expense of future technical debt (that hopefully someone else will have to pay).

I think at least some of this responsibility is on development. They should negotiate and secure the repair iterations for post launch in exchange for making 11th hour design compromises.

[Joeri]

You describe it as a zero-sum game, but it isn't.

Spending time writing documentation does not mean less time is spent coding, it means less time is spent coding that day, but then that is offset by time gained when someone later has to modify it and can finish sooner, thereby increasing the time available for coding other features.

This article struck a nerve with me because I've been at it 11 years, and feel much the same. The one lesson that has stuck with me most is the quality vs speed paradox: focusing on quality makes you go fast, focusing on speed makes you go slow. The reason is that if you cut corners you are continually wasting time fixing yesterday's cut corners, instead of implementing today's feature. You then feel pressure to rush through the new features to make up for lost time, causing more bugs to stack up, and... In extreme cases the project reaches a virtual standstill, spending all resources fixing past mistakes. Usually that's around the time it gets canceled, no lessons learned, freeing up everyone to do it over on the next project.

So, the issue is not one of budget, but of accounting. You have to account for the future cost of debts taken out on the codebase, and when you do you eventually conclude that the numbers tell you not to take out that debt. Convincing a manager of that however ... well, let me just say that it depends on the manager.

[gcb0]

your company gives you bonus per quarter. you have one quarter to impress with new features in production. you don't write code or tests, while team B does. you get 10 features, they get 4. q1 bonus is all your. your code is full of holes next quarter... but now that project is maintained by some junior Dev and you are crunching 10 features on the well documented project on team B because they clearly needed help shipping.

you get the bonuses every quarter.

[lordnacho]

>Spending time writing documentation does not mean less time is spent coding, it means less time is spent coding that day, but then that is offset by time gained when someone later has to modify it and can finish sooner, thereby increasing the time available for coding other features.

Exactly. But when managers don't see it that way, you have the utilization paradox: people are constantly working but constantly behind.

>So, the issue is not one of budget, but of accounting.

Yes and no. The two are intertwined. If accounting and estimation were correct, the budget would be correct on average. Instead, the budget is on average less that what it's supposed to be, because people do not account for everything.

[landongn]

The real issue is communication, not necessarily budget, as the reason software projects I've been exposed to fail.

A project manager can only get you so far, especially if the team dynamic is rotten or even sub-par.

Management is never really the answer to me, even saying that as a coder who splits time as a manager. At the end of the day, if your team doesn't have the right dynamic -- soft skills or otherwise -- I don't think they've got a great chance of success regardless of how much budget or process you can throw at them.

[lordnacho]

This is very true. I hadn't touched on the cultural aspects of why things get managed poorly.

One thing to note is when the guy calling the shots is not a programmer. It's simply hard to see all the things programmers have to do, other than typing out code. It's also hard to see why some guy wrote a bug if you're never the guy who is responsible for it.

Have a look at sports teams. The coaches tend to be ex players, even if at a very low level.

The other thing about manager types is things never go wrong for them in a way that is purely them. When I worked in marketing, I'd never get stuck on a PowerPoint slide for days. Problems were always organizational, and someone else was always involved. Sales guys aren't selling fast enough. Fab guys are stuck with their new process. PR people have the wrong message.

It's very easy as the organizer to just think all the problems are due to other people.

[wpaladin]

Very true. Except that budget is a limiting factor when it comes to communication as well. A team is going to put much less energy and thought into communicating if they have their hands full.

[throwaway43]

This also has implications for your career development as a programmer. I've been working at an outsourcing shop for the past two years and have barely grown as a programmer.

If you work at a place where the client isn't willing to pay for good quality software you will never learn how to make software the right way , simply because there is no time to do it the right way.

[chii]

which is why a software project requires someone with technical knowledge doing the management, and not an MBA or "project manager" who cares only about on budget and on time at the cost of technical quality.

[Too]

If you know the management is not technically competent, the responsibility also lies on the developer to communicate things on their level. If they ask how long will X take, you don't answer 1 day, and then another day to test it. You answer 2 days. If that's outside their budget then forget about that feature and let sales find a new feature they can sell that can be developed within the budget.

This also boils down to the ego thing, many junior developers are proud to give short estimates to show off how quick and good they are at their work. But those short estimates are always always always just the estimated time to get a quick prototype working, where things are configured as you go in the debugger, not something that is robust and will work together with the rest of the application in a customer deployed environment. Estimation is really where you can see the difference in an experienced engineer and a non experienced one.

[yeukhon]

TL;DR - you want to hire people who can adopt and who respect both sides (business, developers)

I work as devops and my boss always says that good operation/support people are hard to find. One can be a super hero in programming, but really bad at customer service or supporting production. Similarly some ops are very horrible at coding, but they are great system administrators.

With that, I have had product owners and product managers who are really excellent at managing team and able to cope the lack of technical skill by absorbing the technical knowledge from daily standup and eventually able to work with the team to prioritize technical challenge. For example this one product owner works in big data and he couldn't ssh without me showing him, but he could go over the pipelines just enough to make me feel embarrass. Of course, if you are on a project long enough you should know how things work in general.

On the contrary I have had really senior technical people leading teams and eventually got fired for their inability to lead.

[enjo]

In a business on-time and on budget are pretty damn important. I'd argue for most software disciplines (note: I didn't say all), probably more important than technical quality.

[lstyls]

It's been tried over and over again. Throwing developers at a project has negative returns at some point.

The truth is that individual engineer skill is the only good bet, and only if within a culture that works.

[collyw]

That's true, if and only if you have decent experienced developers. I work with plenty of mediocre developers, who can code, and not much more. They don't seem to see patterns between things. They don't think in sets. They probably don't refactor their code to make it maintainable for others.

[function_seven]

This is actually in reply to d--b, who appears to be hellbanned.

d--b, you're hellbanned for some reason. FYI.

[d--b]

I think the main issue the OP is trying to describe is not the tradeoff between cost and quality. I believe this is well understood. However the problem that is not well understood by non-developers is the cost of change. Maintaining software is _extremely_ costly, and people simply do not get that.

While everyone understands that maintaining a physical infrastructure like a highway involves a large amount of energy (create an alternative road to redirect the traffic, fix the road, destroy the alternative road), it is not true of software. Because what happens in the code is not obvious, clients and managers have no idea of the effort that would be required to change an existing piece of software without disrupting the current flow of operations.

Let's take a more practical example: The boss asks you to create a datatable in a database to store client information. And according to the first specs, each client has one address, so you go on and create a table that contains client id / client name / client address. Everything works fine, great! Now, you do your demo, and suddenly the boss tells you: hey I've got 2 addresses, can I have my 2 addresses stored in the system? You have then 3 choices:

1. Tell your boss: no there's only 1 address in the system 2. Tell your boss: ok I can put 2 addresses, but then it means I have to split the user table and create an indirection. So I have to rewrite the whole address read/write layer. 3. Tell your boss: sure I can do it. What I'll do is take my first design, add 3 or 4 'additional' addresses to my table (who's got 3 or 4 addresses anyway?) and I'll just have some minimal changes to make to my code.

In most cases, if you're the developer, you'll opt for option 3. You may think in the back of your head that this is disgusting and that you'll change it later, but any of the other two options will make you look pretty bad.

Later on, the boss asks you, hey you know what would be great, is that we can lookup users by their addresses. And now you find yourself having to deal with these additional addresses columns all over the place, and here starts your ball of mud.

I think you see my point. The problem is not a question of money, it is very much a problem of understanding the underlying structure of a project.

[zobzu]

In our current society model we don't really have a "choice". Basically, if the budget is put towards "doing things right" there is very little chance the project takes off, its going to be "too much long term".

Are we doomed to fail? Maybe.