[Pyxl101]

Sure, I believe you that there are certain tasks where years of experience doesn't affect performance much. I could believe that certain kinds of programming fall into that bucket.

How about heart surgery? If you or a family member needed to have a risky or complex medical operation done, would you prefer to use a specialist in that field who has done that surgery for 15 years, or would you prefer to be operated on by a surgical resident who is qualified to do the surgery but has limited experience? Which doctor has better average patient outcomes?

Programming alone is not the only discipline that businesses need. They also require skills like leadership, business sense, product and design sense, etc. There are many kinds of aptitude that one develops by having done something before. Imagine that your programming task requires designing a new Linux kernel API. Two programmers have equal experience, but one built Ruby web applications, and the other was a kernel developer who designed cgroups. Who will have the advantage in designing the new API?

Relevant experience in any area that's broad like business or medicine will tend to make someone better. I believe that you can craft a set of "programming challenges" that will show that people with limited experience can perform as well as those with much experience - I don't think that programming challenges benefit from the kind of judgment and insight and leadership that experience brings. Experience benefits some tasks, but not all of them. Furthermore, knowledge in some areas might be relatively shallow. The Peopleware study purports to be about "knowledge of a platform" according to a blog:

> Once they had six months under their belt, the platform knowledge was no longer the bottleneck in their abilities.

Six months might be enough to reach 99% diminishing returns for knowledge about the platform. I can believe that. I don't think this establishes a general point about the effect of experience on performance. Experience benefits some tasks more than others. I doubt that six months is the point of diminishing returns for heart surgery, for example, and for many types of engineering.

Broadly, I don't understand the alternative belief: are we saying that people do not become more effective with experience at all, for any task? Not even a little?

I certainly study diligently and apply myself every day toward getting better, and I certainly seem to be getting more effective in a variety of ways. But I cannot believe that we are fixed as humans the day we turn 18, and can become no more competent than we were then. It violates common sense. The fact that experience might have diminishing marginal returns after a certain point does not mean that experience is worthless, though it may mean that for that task you don't benefit from hiring someone with any more experience than that. But, to hire in such a specific way as that, you need to know what the cutoff is across every type of expertise that matters. If we conceptualize performance as skill across a range of dimensions, then probably many of them improve with experience, though not all will.

I believe that larger-scope tasks are likely to be the ones that benefit the most from experience, such as: leading a business, or designing a large system, or high-leverage pieces of engineering such as kernel APIs. Programming challenges and other tasks "in the small" likely benefit less after the cutoff. For example, in medicine, I could believe that once you have a certain amount of experience drawing blood, that more experience does not make you much better at finding a vein and getting the needle in properly. A doctor with 15 years of experience will probably not perform much better at that task than a nurse with more limited experience. That's a task with (I'm just assuming) a low skill ceiling. Other tasks, like leading a heart surgery, or leading the design of a large system, have a far higher skill ceiling.

[infraruby]

That's a whole lot of speculation!

Fortunately, there is a better way: http://www.ioatwork.com/selection-methods-almost-a-century-o...

EDIT: but to answer your question:

> How about heart surgery?

I'd prefer the provider that hires doctors based on work-sample tests and not years of employment.

[Pyxl101]

The research article you linked confirmed that job experience (years in a similar job) has positive predictive validity for job performance. Can a hiring manager improve their hiring decision by excluding a factor known to have positive predictive validity?

Interestingly, the study also says:

> work sample tests are the most costly of the three (although likely the best approach when hiring for positions that need specialized skills)

This roughly matches how a lot of hiring is done in practice, with the exception of evaluating candidates based on general intelligence. Unfortunately, candidates do not tend to take intelligence tests ahead of time to include in their resumes (nor would they be likely to as part of a specific interview), so this is not a suggestion that's easy to apply to typical hiring. Instead, hiring managers rely on years of experience, work sample tests, personal interests, and years of education, all which this study reports have positive predictive validity.

What concrete change should a manager of a small business make tomorrow to improve their hiring?

[infraruby]

> The research article you linked confirmed that job experience (years in a similar job) has positive predictive validity for job performance.

Job performance does improve with experience up to a point (evidently six months), so this factor will have some predictive validity just for that, but this does not support comparing five years vs ten years.

> Can a hiring manager improve their hiring decision by excluding a factor known to have positive predictive validity?

Yes, if the manager had given undue weight to that factor (say, by rejecting applicants with little experience, regardless of performance on job knowledge tests), which is precisely what happens whenever managers consider employment history.

> This roughly matches how a lot of hiring is done in practice

No, what happens in practice is that managers say "no unemployed need apply" or ask for the applicant's "most recent résumé" (with dates, of course) and then exercise the sort of prejudice against the unemployed that you displayed earlier.

> What concrete change should a manager of a small business make tomorrow to improve their hiring?

The manager should filter applicants using GMA tests, job knowledge tests and integrity tests, which are inexpensive and have high validity, and then pay the candidates to take work-sample tests. The manager may consider experience (up to six months), but as no more than 5% of each candidate's grade, and this should be monitored by the business owner.