[eminent101]

So many bold claims in this comment and little to no justification.

For what it's worth I've seen pretty much the opposite. I don't know about competent vs. incompetent engineers. But when it comes to experience, I've seen the inexperienced ones giving super low estimates and the experienced people giving larger estimates.

[koyote]

> I've seen the inexperienced ones giving super low estimates and the experienced people giving larger estimates

I have the same anecdotal experience with a possible explanation:

Inexperienced engineers often don't see the greater picture or the kind of edge cases that will probably need to be handled ahead of time. I've often had the following type of conversation:

Engineer: "I think that would be a day's work"

Me: "This will need to interact with team X's package. Have you accounted for time spent interacting with them?"

Engineer: "Oh no, I guess two days then"

Me: "Will this account for edge case 1 and 2?"

Engineer: "Ah yes, I guess it would be three days then"

Me: "Testing?"

Engineer: "Maybe let's say a week?"

On the other hand experienced devs might have their judgement clouded by past events:

"Last time we did something with X it blew out by 3 months" - Ignoring the fact that X is now a solved issue

[roenxi]

> "Last time we did something with X it blew out by 3 months" - Ignoring the fact that X is now a solved issue

This is software though, if X has actually been done before then it doesn't need to be done again. It is already done.

Task X clearly had the potential to blow out by 3 months, and they are now working on task Y that is similar to X. It is a reasonable position to assume that there are other as-yet-unknown issues that might cause it to blow out by 3 months until someone has demonstrated that all the unknown unknowns are also resolved by doing it quickly. That is just basic evidence based planning.

[lesuorac]

I've always found that finding a similar scope problem and how long it took is the best predictor of how long the new problem is going to take.

[c0balt]

Ime, as a junior dev/ops person, there is almost always scope creep and adding padding grants you room to account for the new idea your supervisor/ user thought of when being midway into development. As far as I can tell, my supervisor also assumes my estimates should be padded more because sometimes you might need wait on human i/o for longer than planned (holidays/ sick leave/...).

[skeeter2020]

one thing that I like, that can help, is to add explicit things in the spec that it will NOT do. If you keep this "types" of functionality you can shut down a lot of scope creep: "we need to send an email alert after the job is done." gets answered "we can do that in a future iteration because this says the feature will not include any alerting or notifications, just log to a file and finish".

[gjvc]

But when it comes to experience, I've seen the inexperienced ones giving super low estimates and the experienced people giving larger estimates

is the essence of this thread

[taurath]

A big problem when you're a more experienced engineer is when you have your hands in a lot of things and know the relative priority of stuff and how likely it is that something else of importance will pop up. So you anticipate things getting sidetracked over time, and try to make a bit of a longer estimate, usually to give yourself the slack to do other important things without looking like you're falling behind in JIRA.

Giving an "if I had nothing else going on" estimate can be a big trap to fall into - they will only see the number and judge your performance based on that. This dovetails into the problem that untracked but still important work being thankless in low trust environments - not all work can ever be tracked, or else the time to track that work would take as long as doing the work. Examples: literally any emotional labor, time to monitor, time to train, time to document when its not explicitly required, time to solve little problems.

In the environment where none of this counts because its not quantifiable, everyone with knowledge makes themselves into a silo in order to protect perceptions of their performance, and everyone else suffers. I'll go even a little further to say that companies that attempt to have no untracked work are by nature far more sociopathic - thus far there's basically no consequences for sociopathic organizations but I hope one day there will be.

[bdangubic]

I think general problem on HN is that you can't say something "bold" without people going "nuts" - especially when it comes to estimating work.

In my experience (been hacking since the '90's before it was cool) great developers are great at estimating things. And these are not outliers, all except 1 great developer I've had pleasure of working with over these years has never been "off" on estimates by any statistically significant margin. but you say anything like that here on HN and it is heresy.

My general opinion is that developers LOVE making everyone believe that software development is somehow "special" from many other "industries" for the lack of a better word and that we simply cannot give you accurate estimates that you can use to make "deadlines" (or better said project plans). and yet most developers (including ones raising hell here and downvoting any comment contrary to "popular belief") are basically doing sht that's been done million times before, CRUD here, form there, notification there, event there etc... It is not like we are all going "oh sht, I wonder how long it'll take to create a sign-up form."

I think we have (so far) been successful at running this "scam" whereby "we just can't accurately estimate it" because of course it is super advantageous to us. and WFH has made this even worse in my opinion - given that we can't provide "accurate estimates" now we can simply pad them (who dare to question this, it is just an estimate, right? can't hold me to the estimate...) and then chill with our wifes and dogs when sh*t is done 6 weeks earlier :)

[mewpmewp2]

It really depends on what you are working on. When I did agency type of work, building things you have built before, from scratch, it's easy to estimate. E.g. some sort of e-commerce website from scratch. On the other hand, working in a large corp, with massive legacy systems, unknown domain knowledge and dependency on other teams, it becomes near impossible. What might have been a 2 hour task in agency, might either be completely impossible during our lifetime unless the whole company was built from scratch or take 2 years. You might need 6 other teams to agree to make some sort of change, and you first might have to convince those teams, and then 2 teams will initially agree to it, then pull out in the last moment, or realize they can't do it.

[bdangubic]

In your 2nd example of a large corp I can see myself joining and not knowing anything about all these potential landmines etc… but if I was at that corp for say 2-3 years those would all be known things, no? And hence estimates can come with number of assumptions based on 6 other teams doing whatever it is that they need to do etc…? But I totally agree with your point and would not be disappointed if someone missed an estimate in that sort of chaos.

My comment was not really geared towards chaotic organizations but general sense (especially in myriad threads on this forum) that SWEs think that somehow our profession is so “special” that we cannot possible know how long something will take. I simply do not accept that and personally believe that more likely vast majority of SWEs simply suck at their job. there are roughly 4.5 million of us, how many are really good at their jobs (big part of which is estimation)? probably like 0.4% :)

[WOTERMEON]

I agree with the general sentiment (estimation is part of the work, ppl are not good at estimates, ppl are not good at their job) I also think that in high churn rate companies, or where team get created and disbanded every two quarters, it’s quite difficult to have a mental model on other teams ability to deliver a dependency for your team. And this situation I find quite common tbh

[redleggedfrog]

You can't make accurate estimates that can be used for deadlines for non-trivial work. You can make educated guesses on how long specific things will take, and it might be a pretty good guess if you've been keeping metrics on your past work, including things like vacation days and other similar disruptions as well, and keeping a team together long enough to have solid institutional knowledge on your code-base. And you can respectfully lay these numbers out to a manager in the form of, "This will probably take 1 to 3 days," or for bigger stuff, "2 to 3 weeks", and so on. And the manager can take the sum of all this and say, "The soonest it can probably be done is 3 months, but most likely it'll be 4, with a small chance of a bit longer", or whatever, you get the idea. And then the mangers can set the deadline as they see fit. Now, for some reason, many managers just look at the first number and are done - that's the due date. And then after that they get deer in the headlights treatment, so the worst case becomes the best case. That's on them. If they don't understand that software estimation isn't an exact science they're in the wrong field.

As for software development being special, I really hope that what I've described above is like other engineering disciplines, and we're not special. I don't want to be special, I want to be an engineer, like those who work with aircraft or bridges and what not. I feel like in those fields the concept of estimation is a little more respected. But I'm probably wrong. :^)

I'll mention I've been a professional software developer since the early 90's, not that experience equals veracity. But I've had good success using the system above, and even though the bad managers to good managers was pretty even during that time, the company experienced outstanding success during my tenure (20 years!). In the end, bad managers never last. Good managers, who take reasonable estimates to their superiors, succeeded, where managers who brought "It'll be done July 1st" got doubted because their superiors know it really doesn't work that way.

[bdangubic]

your comment is exactly what I am talking about. you actually CAN make accurate estimates for non-trivial works. try to envision this - I hand you a non-trivial assignment to estimate with a condition that if you meet your estimate -/+ 5% you get 7-figure bonus. alternatively if you do not you get fired. after working 30 years in the biz you tell me which of the two is happening for you?

I worked at two places that gave huge bonuses when deadlines were met (based on “estimates”) and wouldn’t you know it sh*t always got done on time and people got paid.

[leetcrew]

in the short term, people will work crazy hours to hit a date if it's the difference between a 7 figure bonus and getting fired. if the estimate is based on devs working reasonable hours, that's a lot of slack built in. I'm sure they hit the date more often than not in your scenario, provided they control most of the dependencies, but it's not a sustainable approach for delivering features.

"provided they control most of the dependencies" is a pretty important caveat by the way. many times I've seen people get the rug pulled out from under them by partner teams at the last second. it doesn't matter how clever you are or how hard you work. if you depend on something owned by a team far away in the org chart, they can always blow up your project with little consequence.

[bdangubic]

I honestly do not think it is about working crazy hours. perhaps my example can be misunderstood in a sense that if you give someone 7-figure bonus they will inevitably work 20hrs/day if necessary to get there which of course would not mean that they estimated correctly but were off by 12hrs/day :)

as things stand what is my incentive to provide an accurate estimate? if no one can question my estimate and hold me to it (well perhaps they can question it but we as industry have successfully been able to convince everyone that these are just estimates, nothing else...) what is my incentive to be accurate? If like one of the commenters above can say "it'll be 2 to 3 weeks" there is an INSANE difference between 2 and 3 weeks, 33% difference. it's like coming to buy a house and agent says "this house is $200k or $300k but you sign here on the bunch of dotted lines and we'll tell you all about it eventually before you have to cut a check." It is good to be in this industry (and especially if you WFH) - say 2 to 3 weeks, finish in 2 and get a week of working on your wellness (or another job :) )

[yetihehe]

> it's like coming to buy a house and agent says "this house is $200k or $300k but you sign here on the bunch of dotted lines and we'll tell you all about it eventually before you have to cut a check.

Not really with price, but when I've had my house built, the date was overshoot by about 30% too, because of various reasons, like having to stop for winter because some supplies were late by a week several times or my builders had to help teams at other places from time to time (because other teams were late too), not doing anything at my house sometimes for days. So even when building homes (something they do again and again) you can't really put exact estimates.

[redleggedfrog]

That's an interesting scenario you're proposing.

To answer it personally, which of the two, 7 figure bonus or being fired, it'd be I'd quit. If someone is structuring the development of software based on this premise, then they are going to need a different kind of person than me. But I admit I'm probably an outlier here. I don't really work for the money, and my salary is enough, and I don't like undo pressure.

For arguments sake let's say the 7 figures is $1,000,000. To offer that kind of bonus the project is likely going to be a larger one. And I'm assuming my estimate is determining the deadline, so of course I'm making sure it's something I think I can achieve.

But then there are other significant problems with this structure and the likelihood of meeting the deadline, and, more importantly, generating good code and user experience.

- +5% (in ignoring the -5% as no one cares if you're early unless it creates some sort of QA burden) implies a narrow window. On a 6 month project that is ~6 days. Enough that personnel changes or other uncontrollable factors could lead to a missed deadline. One person getting fed up and leaving would be a huge problem.

- The specification would have to be really clear and agreed upon, since there is much at stake.

- Any changes, scope creep, customer requests, could change the development time, and you'd have to have some sort negotiating buffer built in since there is now so much at stake. Otherwise you're going to get literally everything rejected by the developer as they drive towards the deadline (maybe that's what you want, though).

- Is the result worth having? A focus on a deadline, in my experience, tends to shortchange quality. But maybe the deadline is more important than quality.

- And lastly, if that deadline is missed, or worse, something changes the scope of the project and the bonus is not awarded because that led to the deadline being missed, you're going to have some super pissed developers that will not trust such an arrangement in the future.

I suspect you're talking about situations beyond my pay-grade. I've been a meat and potatoes programmer working in e-commerce and integrations mostly, and we don't see 7 figure bonuses. We certainly have had can't miss deadlines that we mostly didn't miss, but mostly those deadlines were due to external factors (API deprecation mostly), or financial considerations, or lastly, arbitrary deadlines set by management. On the latter, those mostly got missed. But that was to be expected as they were not tied to reality.

And I'd second leetcrews comment below of, "...but it's not a sustainable approach for delivering features". Maybe this scenario works once or twice, but it seems like a terrible way to develop software.

But still, an interesting thought experiment.