[jchw]

No offense but reading this comment with the utmost attempt at interpreting it in good faith, it kind of sounds like you're just going off of vibes and can't actually quantify productivity any more than anyone else can. Otherwise... How exactly are you quantifying it? If you had to show your work, could you?

[johnfn]

I dunno, the opposite side seems equally absurd to me. Do you work at a software company? Do you have coworkers? Are you honestly telling me that, gun to your head, you couldn't say that some are more productive and some are less? That if you had to choose a co-founder for your next startup, that you would have NO idea where to begin, that any of them would be equally as good as the next?

[jchw]

I mean look, some people seem to be extremely productive, and sure, I could say that those people are almost surely more productive than others. It's still genuinely very hard to quantify if those people are actually vastly more productive, or if it just looks like that because they produce more obvious artifacts of their productivity. Hell, what is productivity, is it more productive to fix 1000 bugs or to be the tech lead on a product that reaches 1 million ARR? Does it matter what bugs?

I stand by what I said, whether people like it or not.

Including the part about going off vibes. Please explain how this "you can just kind of tell" mentality is not literally just going off of vibes?

(And I wouldn't choose a cofounder only based on productivity, anyways. I'm not even sure that would be the main criteria.)

[johnfn]

How would you choose a cofounder? I would call whatever metric you use to determine "would that person be a good cofounder" productivity. (OK, sure, let's narrow it down and say that you were tasked with choosing a purely technical cofounder; perhaps in this hypothetical you're helping your non-technical friend find one?) Do you still believe you couldn't do better than chance?

[jchw]

Well, a lot of the factors explicitly have nothing to do with general productivity at all, such as alignment. The goal with regards to selecting for productivity would be "someone who seems productive enough" but getting hung up on exactly how productive feels like a mistake, as aside from being unquantifiable, that isn't going to be the only or probably even main factor that decides the fate of the endeavor. Productivity is not the most challenging problem in software development. In my opinion, as far as factors for success for a technology endeavor goes, even overall technical competence winds up factoring in fairly low at the end of the day. What's important is meeting the threshold you need, not getting as high of a score as possible.

I should clarify that I actually agree with you if what you're saying is that the best measure of productivity we have is literally just going off your gut, but my argument is that this gut feeling is terrible. This is in large part because humans are biased, our gut feelings are swayed by things that simply shouldn't factor in. We tend to have more positive opinions of people that we think are like us and we sometimes wind up having negative opinions of someone based on stupid things like disagreeing with them on something that is ultimately irrelevant to whether or not they are productive.

And to me, the ultimate nail in the coffin is really in the question of what really constitutes productivity. Productivity is supposed to measure the efficiency of producing something, which already has pitfalls in and of itself when dealing with things that do have discrete, measurable indicators of progress, but programming doesn't, it's not even always obvious if progress is forward or backward sometimes. What's most important, performance optimizations, disaster planning, features, robustness, minimizing resource utilization? The best answer you can generally say is "It depends," and moreover, everyone will have a different set of competencies they're best at, so a person doesn't really have some single "productivity value" you could summarize them with. Yet, all of those things are pretty important for any serious technology organization, so you would want people with a variety of different affinities.

At that point, it starts to beg the question of whether or not attempting to directly measure software productivity is a worthwhile endeavor. I'd argue not.

[robertlagrant]

I suppose the question is: can you supply a word you would use when deciding on who to hire for a technical role? Does anyone off the street contribute the same amount of that word, or would you be selective?

[jchw]

Isn't it obvious? Hiring people who are actually competent is hard. It's not just me, major corporations have found the same issue.

I do believe that you can use basic tests to determine whether someone has more technical competence than some random guy off the street, but it only works up to a point. If you try to test deeper and deeper knowledge, you might create a mirage where someone who isn't very competent appears competent because you just happen to hit on strong spots on their very sparse experience. (My imposter syndrome reasons that this is why I passed the Google interview so easily some years ago.)

But for example, interviews don't even really bother trying to determine any direct proxies for productivity. Usually, they just stick to trying to determine technical competence, communication, ability to work on a team, and evaluate their history of technical accomplishments. A list of accomplishments is evidence of productivity, to a degree, but not having a long list is not evidence of a lack of productivity, and neither will tell you what will actually happen when you hire the person. References will at least give you someone else's gut feelings (or lies) regarding someone's productivity, but any reasonably competent person is going to have people who can vouch for them even if their productivity is actually not very impressive. It's not like there's some huge punishment for embellishing someone when you're being interviewed as a reference for them.

In the context of hiring people, gut feelings are probably the best thing we have, but they're subject to horrendous bias. Even if you are highly enlightened and can recognize your own biases with a great degree of humility, this is not generally the case for most people. Because of that, Google's interviews have a lot of layers of abstraction designed to eliminate bias from the process, but then again, they also wound up doing a study where they hired people who were ultimately turned down for the job and found that those people had around the same chance at succeeding at Google as the people who were hired. (Can't find the source for this because Google Search is useless nowadays. Maybe their hiring process is to blame.) And yet, there's no doubt that even with this in mind bias will still impact the interviews, because the interviewer does ultimately have to transcribe the interview and they can choose to omit or paraphrase things in a way that makes it look worse to the committee overseeing things; likewise, you can "correct" what the person is saying if you felt it was "close enough", or omit entire segments that looked weaker. Sure, you're not supposed to, but I would bet you 10:1 that even people not intending to be biased wind up doing this. Maybe they're second-guessing themselves when they do it: was it my fault they didn't answer better? Were they saying it right all along and I just wasn't understanding?

I was actually involved in a lot of interviewing and hiring especially early on in my career. I still believe gut feeling was the best instinct I had, but there was a time when I didn't agree with a hire and was proven horrifically wrong very soon after. Granted, that mostly comes down to how you evaluate someone's technical competence, not necessarily productivity, but I think the point stands either way.