Why do we need to measure productivity? No one seems to ask that question. To me it seems that trying to measure the individual (people) parts of a system in some quasi-reproducable and transitive way does more harm to the total output of the system than good.
Do we measure trust (upon which all business is based)? If we relinquish individual productivity from our “measure all the things so we can improve them” obsession to the soft realm of the immeasurable, we might magically stop wasting so much time on stupid shit and accomplish a lot more meaningful work.
I am with you, measure results and assume the team got it done with minimal effort. If it is for headcount then ask if you can get the same results with less headcount. If you want hourly workers who don't give a shit about results, hire those. If you want result oriented salary workers then hire those. The problem is every manager thinks they will be the one person to figure out a middle ground without consequences.
Micromanage here, red tape there, metrics-squeeze salary people and blame anything but your management tactics when things go south. I have been fortunate enough to have had some managers who knew better than all this nonsense though.
> Do we measure trust (upon which all business is based)?
Yes, absolutely. It's difficult to do this in customer surveys because you need a motivation for the customer to engage in a feedback process (e.g. a product review, or a questionnaire after a support call), and then the feedback is mostly about that specific event. But focus group research will absolutely try to measure trust either implicitly (e.g. from sentiment analysis) or using explicit questions.
I also know of at least one company that measures (or at least, tries to measure) "leadership trust" among their employees using surveys.
So would you just fire the entire team if a product does poorly?
Many individuals don't like startups because of this risk, most startup only get 1-2 chances to get productivity right. Very few waste time measuring productivity.
Start-ups don't get 1-2 chances to get productivity right.
They get 1-2 chances to get PRODUCT right.
If they don't get productivity right from the get-go, they only get 1 chance instead of the "1-2 chances".
That's the risk most individuals don't like about startups, if productivity is not a centerpiece of their operation, the leadership pushes for the only other solution they believe: long hours in inhumane working conditions.
I don't believe there's causation between a start-up productivity and their product success, but I would love to be proven wrong.
We undervalue qualitative measurements when talking about this stuff. But in real life it is all there is. If you did everything with a measuring tape, you could do nothing (not even walk to your car!). If you care about nothing and close your eyes, things will get disorganized and chaotic and out of control.
This doesn't mean measure nothing, but don't try to get some perfect measure of productivity. At least not in creative work like software development.
We measure trust all the time: everyone you work with keeps track of whether you tell them the truth and meet the commitments that you make.
People also assess your productivity and the quality of your results in deciding whether to work with you, to ask you to join their team, to hire you, etc...
I would be surprised if you don't try to assess your own productivity and the quality of the results that you deliver. Neither of these may be easy to measure but I don't think you should be ignoring them.
>Why do we need to measure productivity? No one seems to ask that question.
because the people asking the questions aren't the ones who actually do the work. And timelines are everything to those people (which admitedly falls behind a lot in software due to the nature of the work).
>Do we measure trust (upon which all business is based)?
Glad you asked. Because "trust" is one of the last ways companies can get around being discriminatory in at-will employment without facing lawsuits. In other words, it directly benefits those in power not to directly measure "trust", but quietly base it on the biases of any given manager.
Unless you're in some stack ranked program, lack of trust is the easiest way to be first to be laid off, or on the slower track to be put on a PiP (officially or not). Sadly and ironically enough, "productivity" is one of he few ways to counteract this bias. Having a hard trail of evidence that you met every and all goals, a hard trail that these were the goals to begin with.
I could go on about how you gain trust, and indirect trust or trust via authority, but I think you get the point.
When you need to do a job and have a certain amount of resources available you need to measure the productivity of your process. If you don't do that you'll waste your resources.
In business this means that if you have a machine that produces X widgets per hours and costs Y dollars to run, you can estimate is it worth running it at all or it's not economical to do that.
With teams of people managers/owners solve the same problem - I have a fixed amount of resources and need to allocate them properly. No measure = no good allocation = going out of business.
The reason capitalism creates more and cheaper goods that socialism is that resource allocation problem is solved by every participant in the free market (or mostly free market). Without measurement, the owners can't allocate the resources properly and they'd go out of business.
If you have worked in a team with badly performing members who don't pull their share of the load this creates imbalances in the team and the best performing members of the team leave, because they don't want to deal with the laggards.
Because it's not observable; it has to be inferred. Every employee affects the finished product through a complex web of interactions. As a latent variable, we can only hope to gain a noisy estimate.
Companies don't bother estimating worker productivity themselves. Rather they let employees make their own case, based on their work, using whatever data they can scrounge.
Creativity follows a different pattern than highly repeatable line work. Software engineering is an inherently creative pursuit, especially when paired closely with a dynamic product function.
Definitely recommend Rick Rubin's book The Creative Act for a detailed glance into how wildly off the attempt to systematize and measure productivity is, in a creative context. https://www.amazon.com/Creative-Act-Way-Being/dp/0593652886
> Interviews will transition to be more real-world scenarios, perhaps within the customer's actual codebase, addressing a genuine problem the customer faces—possibly even compensating the interviewee for their time
The mechanics of this idea suck for everybody.
Sometimes you have a candidate who is up to speed on your stack or one of its dependencies and can go from idea to implementation in a short time period but otherwise the math doesn't work.
A week of time with a mentor for one day over that time is probably enough to do a 2 day task if there are no difficulties when you consider lack of knowledge of the code base and interfacing with a new code environment.
So best case scenario you pay 3x for a feature whose mental state won't stick around and the interviewer has to find time to do a week of work while juggling their existing job (which could be job searching if nothing else).
You could say "but you would hire them after" but if we are talking about that late in the process probationary periods make more sense.
If you aren't sure they are a good fit (say 50% certainty) overpaying for a potential solution is bad and then wasting that much time on a solution also doesn't make sense.
That is why most practical things are less than a day of effort and not impactful on the product to avoid weirdness around NDAs/Copyright/proper payment.
And if you think meaningful work happens more often than every two days I wonder what software company you are working for... Breaking down tasks to 1 hour slots isn't what I would call a good use of resources...
Lot of poorly thought out stuff here, I suspect the author is young and lacks real world experience running a business.
> If 10x engineers truly exist, why do pay scales intra company not cover a 10x spectrum?
The value you produce presents a cap on how much you can earn, not a floor. If you want to capture that value you need leverage, high end SWEs usually don't have that leverage. The statement is wrong anyway, at companies like Google, pay ranges from 200k-10million+.
> Interviews will transition to be more real-world scenarios, perhaps within the customer's actual codebase
Many good companies already do this, it doesn't actually change the interviews that much. All interview questions my company has are taken straight from the codebase - the end result is a standard system design + leetcode style interview because we actually need to solve these problems somewhat regularly.
Other options listed that lengthen the interview process from the current 6-8 hour standard are off the table at any companies targeting top of market talent who won't put up with longer than standard interviews (makes it hard to get companies to compete for the interviewee).
Companies not targeting top of market talent will almost always ape top of market won't adapt because they are not generally tech first and thus simply want to adopt "industry best practices". The organizational incentives to do this are incredibly strong.
Small tech-first startups are usually the only ones who can muster the organization will to do something more original recruiting wise but will necessarily stay a small part of the market.
> Interview performance (at least for those hired) definitely does not correlate with on-the-job performance
Citation needed, every company I've seen try to do a large scale assessment of this has found that interview performance is positively correlated with on-the job performance[0]. (I've seen some actual data + studies, but not sure any of them are fully public). What does the author have to offer against this?
I'd encourage the author to think much more about the incentive structures of the actors involved here and try coming to some new (more interesting) conclusions.
>Entire team goes around and answers the question: who do you want to leave?
This sounds like it might work as a one-time thing, but just imagine the second-order effects. Your annual reviews literally become Survivor, how could you ever prevent backroom alliances?
Currently I’m following the idea that it starts with good work priorization. With other words: What work shall an organization/team/individual actually execute? Which tickets shall be picked from the infinite stream of tickets? How do you know that the right tickets are picked? Unless this questions can be answered, measuring the individual performance is completely impossible in the short term. One can „work hard“ and produce a lot of code, write many tests, do many tickets with low ROI and still achieve subpar results. Thus the measure of productivity must truly capture the added value of one’s work. And there is no „one size fits all“ solution.
Long term the differences in productivity are clearly visible.
From the OP: “every potential metric..is woefully inadequate”. How many years do we keep trying before realize there is no such thing as individual productivity.
My point is more radical than “we all have the same productivity.”
The very concept of individual productivity is incoherent.
Putting aside the obvious Goodhart's Law implication productivity isn't just based off of the "components" be they human or otherwise but also the fundamental goals which control the end product.
If I was an utterly mad billionaire who believed that I could hire top software talent and have them attempt to store physical objects in data. Not 'encode the schematics' or even 'encode the complete physical details' but such that it was possible to generate some hidden combination. Such an endevour would not be productive but the fault doesn't lie in the employees.
I think two questions are only needed: are people, teams, and organizations working on what they are supposed to be working on on the scale of weeks to months? Are they hitting acceptable milestones? And when I say milestones, I don't mean time-based ones, which are more often than not arbitrary.
>when I say milestones, I don't mean time-based ones, which are more often than not arbitrary.
artbitrary, yes. But also what the entire business world revolves around. quarterly earning calls, monthly finances, yearly budgets. It's arbituary, but those are period they pick to determine a bunch of stuff above the product itself.
I like the article's premise. I would like some argument for why we'd suddenly get better at measuring something that has escaped us for at least 80 years when knowledge work became more common.
> My suspicion is that [...] there will be a surge in our capacity to measure and evaluate real-life work performance.
It is quite easy. Just apply the income = productivity hypothesis that economists seem to love.
Whoever you pay the most, is the most productive person in the company. Yes, that means you, the CEO, should pay yourself the most because you are the most productive employee...
I guess to all the whiteboard coding with the recursion (e.g. Fibonacci sequence). On a contrary, I think it is present in many places where you have to dive into any sensible, tree like, structure. Then, it's a mess once it turns out person has no idea what recursion is.
They are correlated until you admit to using them in the performance process. Then they’re far too easy to game and become useless. What we haven’t found is an adversary-resistant measure of productivity.
They are absolutely not correlated. Engineers often spend some quarters investigating and fixing scaling issues. This work does not need to a high number of locs.
If your work is merely adding some feature factory or bug fixes, maybe there is a correlation.
If that's how we're getting paid, I think its time to fire all junior devs at the least, and then get onto the slackers and people who don't really do their job
Do we measure trust (upon which all business is based)? If we relinquish individual productivity from our “measure all the things so we can improve them” obsession to the soft realm of the immeasurable, we might magically stop wasting so much time on stupid shit and accomplish a lot more meaningful work.