[zeroCalories]

I think the main problem is that measuring a lot of these things is incredibly hard. The post mentioned laptops and build time and it reminded me of a blog post[1] trying to figure out if it's worth upgrading to m3 macbooks. I encorage you to read the post yourself, but my impression is that the whole project was a huge waste of time. I suspect most attempts to measure impact will end this way. I also think a lot of the recomendations on fighting frupidity are seriously flawd. For example letting developers decide isn't any garentee that you've made the right choice, but it is a garentee that you've made an expensive one. Unless the developers are basically paying for their own tools like in a startup, this is definetly a way to piss away a companiese funds on developers that are more concerend with their own career than the profitibility of the company.

[1] https://incident.io/blog/festive-macbooks

[lawrjone]

I'm the author of that MacBook blog post. I wouldn't say it was a waste of time, in that it helped us feel very confident about the ROI for upgrading, and the results have been really meaningful for our team.

I wouldn't agree exactly with the article, in that while it's very easy to start making decisions that are genuinely ROI silly (all companies make them, my current one not excluded) there is a balance between just paying without question and adding friction that encourages good decision making.

But in our case, getting the data wasn't too much effort, and helped inform us for a subsequent batch of hardware purchases. It ended up representing about $50k of spend that we'd like to allocate well, and took me a couple of days to investigate and write-up: my day rate means that was well worth it.

[zeroCalories]

Unless we can tie compile time benefits to an increase in revenue(which maybe you did?), this is not a quantitative decision, it's more of a gut feeling. Measuring the impact of tools on productivity is hard, and IMO borderline pointless. This is my main contention with the article and your post. Perhaps you tried to do some napkin math on time saved * hourly cost of an engineer to determine productivity gains, but that's a very shaky model. Personally, when my compile takes longer than a few seconds I'll just go read some emails or browse the web. The few seconds gained are just washed away by me reading and replying to an email. I suspect most people are like this. I would need to see some large scale A/B testing on the number of PRs or something to know that this has saved substantial time. My main concern with collecting and using stats like you did is that I feel like you’ve gained unjustified confidence in the project. All that said, I still enjoyed reading your post, and I don’t mean to denigrate your work. I may have chosen words stronger than I can justify because I may have missed something in your post, and knowing that there is a difference between the M1 and M2/M3 laptops is definitely valuable for making the decision.

[praptak]

Measuring the fallout being hard is one problem. The other is organizational - when the unit that owns the savings is separate from the one that bears the costs, they have no motivation to consider the costs.

Their spreadsheets look good after all.