[eesmith]

I'm pretty sure I heard the same point made 20 years ago against using IDEs, with their auto-complete and tool-tips and style feedback.

And 40 years ago in the debate over letting kids use a calculator in school instead of calculating by hand.

A footballer follows a very constrained set of rules. If the footballer were allowed to use a car then 1) it would be easy to score a point, 2) the field would be ruined, and 3) people wouldn't pay to watch or support the team.

If a programmer uses ChatGPT to get a handle on a task with a new API, and saves a day of futzing around, how is that NOT relevant to job performance? (For the sake of argument, let's say that experiment showed that API doesn't scale well enough, resulting in a decision to scrap that approach entirely and use a different API.)

[emodendroket]

We’re not talking about whether you should use them at work; we’re talking about whether it makes sense to have an evaluation where you don’t use them. Closed-book exams are similar. There’s no real-world situation where you wouldn’t be allowed to refer to whatever materials you like, but the evaluation uses somewhat unnatural circumstances to gauge how well you’ve assimilated the material in a limited amount of time and with a consistent process that’s fair for everyone.

[eesmith]

"It makes sense" leads us back to the linked-to essay, which argues that live coding a solution to "arbitrary and unrelated topics like creating a script to handle scores for bowling or something equally irrelevant" do not make sense as a way to

If the evaluation doesn't make sense, then the conditions placed on the evaluation don't really matter.

Similarly, "assimilated the material" only makes sense if the live coding interview really does cover "the material". To use your analogy, measuring a football prospect’s cycling times aren't that good of a test of football playing skills. I mean, yes, there's some overlap, but there are more useful ways.

And one of the example tests was "handle scores for bowling", which is far from most work-related issues.

"consistent process that’s fair for everyone."

The author addressed this idea at several points, including "Any belief that a live coding interview is a consistently reliable way to make an objective assessment represents willful ignorance at best."

Picking a name out of a hat containing potential employee names is also consistent and fair.

Just because it's easy to measure doesn't mean it's an effective predictor.

[emodendroket]

I suppose I fundamentally do not agree that the skills are unrelated. I think they are the same skills, but applied to a much smaller problem that fits in 45 minutes. The author does not make any case that the smorgasbord of alternatives he offers are any better. And none of this has to do with your “well I can just Google a well-known problem like this in real life so it doesn’t matter” argument.