[wschroed]

I prefer studies over anecdotes and found this: https://www.amazon.com/Accelerate-Software-Performing-Techno...

According to this study, you can measure a team's progress and ability with:

* number of production deployments per day (want higher numbers)

* average time to convert a request into a deployed solution (want lower numbers)

* average time to restore broken service (want lower numbers)

* average number of bugfixes per deployment (want lower numbers)

I am curious about other studies in this area and if there is overlapping or conflicting information.

[flipchart]

Haven't read the linked study, but it seems like all of those metrics (except time to restore broken service) can be very easily gamed - thinking about the recent SO blog post https://news.ycombinator.com/item?id=25373353

[rvdmei]

Why would you want to game metrics that honestly can help improve your business outcome? The study is worth reading and to put it simply: the metrics seem to make sense anyway and if science proves that they do, that’s even better.

[wschroed]

Though I posted the study, I do want to see more work in that area to repeat the observed effects or to refine the KPIs. For example, I am personally skeptical of the notion of "number of defects per release" and want to see an exploration of "amount of time spent on defects per release".

Science doesn't tell us anything (that would be authority or religion), and it doesn't prove things. It is a process that can assist us in logically determining what is NOT true about a cause-effect relationship to the point where we can make more accurate and practical predictions about those causes and effects. More experiments refine the current beliefs.

[logifail]

> Why would you want to game metrics that honestly can help improve your business outcome?

Q: If your manager imposed OKRs/KPIs on your team, especially if there were a financial incentive linked to achieving "results"/"performance indicators", how would you feel?

Note that what is good for the business may or may not have much to do - at least in the short to medium term - with what is good for the employees.

[rvdmei]

Definitely a reason to try and game the metrics (and probably easy too). In my experience the metrics make sense when you use them to improve as a team, I would actually try to avoid using them as OKR or KPI.