[rufflez]

In my experience, metrics are a giant waste of time for engineers and engineering managers. Project managers and folks who depend on the fruits of engineering labor on the other hand love these metrics. It is simpler (and wrongheaded) because who wants complications after all? Look at the trivialization in this article for instance - "tasks come in, and software comes out". Oh really? I had no idea!

Tasks vary drastically from project to project,including due to varying requirements, ambient software environment, technological changes and a host of other factors. Metrics are a way for project managers to pass the buck on to the engineering teams and nothing more.

[tra3]

You gotta have some way of answering the "how long is it going to take" question. There's no better way of predicting things than looking at how long things have taken in the past.

Unfortunately what everyone forgets is that historical information only applies if you are doing "a thing" that is similar to what you've done before.

[tadfisher]

One way to address this problem is to not ask the question, but to work with your teams to set goals for a longer period of time such as a quarter. Then structure your organization such that it doesn't depend on the exact timing of work completion.

[cortesoft]

How does a longer period of time help with estimation? Trying to estimate what you will complete in a quarter is even harder than estimating what you will complete in a week… errors compound.

[AdieuToLogic]

  How does a longer period of time help with estimation?

I interpreted the GP's statement of:

  One way to address this problem is to not ask the question, but
  to work with your teams to set goals for a longer period of time ...

To mean, "don't focus on estimating each individual task, but instead set midrange objectives." Which contextualizes the GP's suggestion of:

  Then structure your organization such that it doesn't depend on
  the exact timing of work completion.

To imply avoid the desire to "estimate what you will complete in a quarter" and instead focus on delivering prioritized stakeholder value.

[cortesoft]

If you are setting “midrange objectives” for a quarter, you are basically saying “I estimate I will complete this much work in the quarter”, because you are estimating you can complete enough work to meet the objective in a quarter. Either the objective is easy enough that the estimate doesn’t have to be that accurate to still meet it, or the objective is not specific enough to require an accurate estimate.

I actually mostly agree with the comment I was replying to, I just think the “set goals for a longer period of time” was not filling in the whole picture. You can’t just set the same types of goals you had when you were making sprint estimations, stretch them out to be quarter estimations, and expect the problem to go away.

[AdieuToLogic]

I think the difference is in plurality. Setting goals for a quarter (or month, or year) to me is more of a team thing/focus/discussion rather than an individual's estimation of what they can do. And I completely agree with you that stretching a sprint into a quarter makes no sense if it is only for estimating one person's work.

Given that so much of what we do as developers typically depends on more than just ourself in an effort, I agree with what I think tadfisher is expressing; realistic delivery planning can be achieved by focusing on "teams and goals for the next few months" and cannot be by trying to answer "how long will <insert feature never done before> take?"

[switchbak]

It really comes down to: can you provide the value we need in this area. That value might be "validate experimental research is implementable", or "perform a spike to de-risk a future effort".

There's a fog of uncertainty that gets impenetrable past a certain time horizon. We need businesses that are capable of forging ahead into the unknown, being honest about the level of confidence given our knowledge at the time. I don't see that approach coming out of MBA schools, and I think Elon's approach is in marked contrast to that.

In many ways old school grassroots agile has penetrated the lower levels of the org (to some degree), but the upper levels haven't changed in decades, and this impedance mismatch is most visible in the rigidity of planning (which is also forced on them by outside pressures).

[djbusby]

Q: how long will this take?

A: what is your budget? What other work will you sacrifice?

[doctor_eval]

> You gotta have some way of answering the "how long is it going to take" question.

Code (implementation) is an integral alert part of the software design and planning process, and you can’t know how long something will take until that’s complete.

While estimation seems to be a shibboleth of certain types of project management, the fact that implementation time estimates are rarely accurate - and often significantly wrong - suggests that they are not as vital as we’re lead to believe.

[tra3]

> While estimation seems to be a shibboleth of certain types of project management, the fact that implementation time estimates are rarely accurate - and often significantly wrong - suggests that they are not as vital as we’re lead to believe.

I disagree with your conclusion. Just because we're terrible at estimating doesn't mean it's not important.

"100% uptime is rarely achieved, which means it's unnecessary and unimportant".

> Code (implementation) is an integral alert part of the software design and planning process, and you can’t know how long something will take until that’s complete.

McConnel talks about it in "Software Estimation". The only way to accurately predict anything is to base it on your previous experience, which I allude to. Unfortunately (or fortunately, depending on how you look at it), even projects that looks similar on the surface are more frequently dissimilar than similar.

Regardless it's useful to be able to answer the "how long" question at least a coarse level to see if it's worth doing something.

[doctor_eval]

I understand that we disagree, but Eric Evans also talks about how implementation is an inescapable part of design, as does Jack Reeves. So it follows, for me at least, that estimates can never be accurate.

Recently posted on HN was a link to a post at Lunar Logic where estimation is basically broken into 3 categories: “NFC” (no clue), “TFB” (too big) or “1”. They explain why here [1].

This has been my experience over more than 30 years of building products. The duration of any given feature/story is effectively unknowable, but we can make a decent guess as to it’s achievability, we can get a decent idea of the rate at which stories are completed.

The main problem I have with more detailed estimation is that the estimation process itself consumes significant engineering resources which IMO should be assigned to actual engineering.

To your specific points:

> "100% uptime is rarely achieved, which means it's unnecessary and unimportant".

This is specious. As an industry our estimates tend to be wildly wrong, and despite decades of dealing with “the software crisis” they are not improving. My position is that this means that we’re spending a lot of time estimating and then more time dealing with them being wrong. It’s just a waste.

> Regardless it's useful to be able to answer the "how long" question at least a coarse level to see if it's worth doing something.

This seems to be the wrong way to do it. Surely the approach should be to ask what value this “something” has to the business, and then determine if it’s technically feasible within that value.

But that would mean more work for the executives, and quite frankly this, in my direct experience, is not the kind of responsibility they like to take on.

[1] https://blog.lunarlogic.io/2016/how-we-estimate-at-lunar-log...

[coffeemug]

You don't ask "how long is it going to take?" You set a deadline and let the black box do whatever internal negotiations it needs to meet it.

[fooster]

Imagine if flight planning worked like that. We are going to get to Paris from New York by 8. Instead the plane lands in the ocean.

[tra3]

That's bleak. Where does the black box start? CEO to CTO? CTO to VP of Engineering? VP to Director? Director to Team Lead?

[ip26]

Or, even more difficult, are you on track to deliver your twelve month goals? Hard to know, and easy to lie to yourself.

[throwaway787544]

I am an engineer, and I rely heavily on metrics, because my job requires quantifying that a system is working correctly. It is impossible to do that correctly without metrics.

Similarly, managing a product team is also managing a running system; that system is just made up of meat sacks whacking plastic buttons with their bony protrusions, rather than computers humming away in data centers. A product manager still needs to quantify that the system making that product is working correctly, and metrics are essential. Otherwise you will only be guessing as to how the system is working, and those guesses will be much more haphazard as the system grows.

[bergenty]

The hard truth is metrics work. I can quantify someone’s performance and if they’re not doing well put pressure on them to do better or hire someone else. Velocity is a pretty good measure of someone’s performance especially if the estimating is done in a sprint planning session with the entire team.

[jdlshore]

The harder truth is that metrics only appear to work. As soon as you use metrics to judge performance, people will start gaming them. They’ll do whatever it takes to get a good score, regardless of whether it’s good for the company or its customers.

The metrics “work,” in that they’ll go up, but the things you don’t measure will get worse, often (eventually) catastrophically.

In the case of velocity, you’ll get people taking shortcuts and sacrificing quality, both internal and external, so they can juice their numbers. The outcome is technical debt and arguments about what it means for something to be done, resulting in slower progress overall.

Source: I’ve been consulting in this space for a few decades now, and have seen the consequences of using velocity as a performance measure time and time again.

(Other metrics are just as bad. See Robert Austin, Measuring and Managing Performance in Organizations, for an explanation of why knowledge work like software development is incompletely measurable and thus subject to measurement dysfunction.)

[mlyle]

The truth is somewhere in between.

Metrics are great to diagnose the overall process and get a sense of what's going on that can be superior to our qualitative feels about it.

And metrics can also spot an occasional outlier or performance problem. Used sparingly, this does not encourage juicing the numbers.

[tremon]

Yes, that is true. But metrics cannot pinpoint the cause of the problem, which is where the engineering approach fails. You cannot accurately measure the stress levels of individual developers like you can with force plates. You cannot accurately measure developers taking shortcuts in their code like you can measure gear slippage. Similarly, you cannot accurately predict the critical load of a particular configuration of developers like you do with beams in bridge construction, nor can you accurately measure the weight of a feature request like you can weigh a vehicle. You cannot measure the "velocity" of two developer teams working on a different codebase and then assume that the metrics are comparable just because you're measuring the same quantity.

The only way software development metrics are useful is to get an indication of the performance over time of the same team working on the same codebase. That should give some indication of the overall trends, but when the numbers start going down, how will you accurately determine the cause of that? Will you treat the developer team as a black box and insert more probes, or will you talk to them and rely on their qualitative assessments after all?

[jdlshore]

Using data and metrics for analysis and self-reflection is great, when used thoughtfully. The problems arise when they’re used to judge performance—or even perceived to be used to judge performance. That’s why they’re so tricky to use well. You have to set up a situation where it’s systematically impossible to abuse metrics, typically by putting the data and analysis/judgement at the same level, and only reporting aggregated/anonymized results and qualitative conclusions rather than the raw data.

Some people don’t know how to manage without measurements, punishments, and rewards. It’s a correctable flaw. Measurement-based management is called “theory X” management, but knowledge work needs “theory Y” management. There’s a lot of material out there on how to do it, including a section on it in my book.

[geekjock]

The hard truth is that using velocity to measure performance will guarantee that developers bias their estimates to ensure they make themselves look good, thereby making your "estimates" useless.

Anyone who's built software knows that estimates are guesses and are bound to change once the work actually begins (not to mention if scope changes midway through).

Additionally, velocity does not "count" work happening outside of tickets, e.g., helping with a customer support issue, assisting another developer with legacy code, reviewing other people's work, spending time on feature planning.

[mandeepj]

> Additionally, velocity does not "count" work happening outside of tickets, e.g., helping with a customer support issue, assisting another developer with legacy code, reviewing other people's work, spending time on feature planning.

Your capacity is adjusted accordingly. In my team an engineer who's going to work-on production support, would not have his capacity accounted. Similarly, if an engineer has a lot of cross-team work going-on then we'll reduce his capacity as well, and so on.

[bosswipe]

Yuck, I'd quit your team. That level of micromanagement is demoralizing and it hurts the company. "Sorry support, I have zero autonomy. I can't talk to you unless my manager adjusts some number in some spreadsheet"

[mandeepj]

Sorry! I'm not doing it in just my team; that's how it's across hundred of teams in my org.

I believe I was not clear: it's a rotational support - where every engineer is encourage to spend a sprint worth of time in every quarter on working on support tickets. You've complete autonomy - you don't want to work on support tickets, it's alright. No one is forcing you. You want to spend next two weeks on a training or self-learning - go for it.

[AdieuToLogic]

  > > Your capacity is adjusted accordingly. ...

  > That level of micromanagement is demoralizing and it hurts the company.

The micromanagement identified is not in the type of work which is most appropriate to success, but instead in the "detailed capacity accounting." This level of "accounting" does not convey trust in an engineer, thus demoralizing them by way of eliminating their autonomy (freedom from external control).

Encouraging engineers to "spend a sprint worth of time in every quarter" or the "next two weeks on a training or self-learning" is laudable, but does not qualify as providing autonomy or the lack of micromanagement. The reasons why are A) the decision is still solely yours and B) clearly time-driven instead of collaboratively prioritized.

  You've complete autonomy ...

Not if you decide when, how long, and with what fixed frequency someone can work on tasks which impact "velocity."

[rufflez]

This is exactly what I would not want my team to suffer. They need to feel inspired, and those who are not inspired will know it themselves, and so will everyone they work with. I don't need a spreadsheet to tell somebody's commitment

[fnordpiglet]

I’m glad I don’t work on anything you’re responsible for. I’ve made a great career out of hiring the people folks like you burn out and frustrate.

I’ll bet $5 that people conform to your model by working longer than necessary and your “velocity” is considerably lower than it could be, despite being measurably consistent.

[bergenty]

No, I’m very insistent on not working late, almost ever. I know the velocity isn’t bullshit because having been a developer for years I know the estimates are reasonable. I also had a 100% retention rate on my team this year with a sizable team and my company paying salaries only a little bit above average.