[georgemcbay]

This was my immediate concern as well, though (to pre-answer the question Thomas already asked you) I don't have any good off-the-top-of-my-head suggestions as to how to stop it.

I think this sort of kick in the pants is exactly the sort of thing that has to at least be tried to sane-ify hiring in tech, but I'm oddly fascinated to watch the ways in which little bits of status-quo inertia and company politics may act to sabotage it.

It is very easy to imagine companies not really "getting it" and seeing it the way they see github contribs or stackoverflow scores or whatever other metric that some do use as a signal, but which very rarely actually gets someone in the door or even on a particularly fast track.

In an ideal world, any sort of testing like this plus a screen to make sure you're not a psycho, a narcissist or just a general pain in the ass on a personal level should be sufficient for virtually any programming job, but there are lots of obstacles without obvious technical fixes: HR dept politics, the hazing culture ("We all did this stupid interview thing, so should the new guy/girl!"), general distrust ("nobody ever got fired for hiring someone using the old method"), etc.

I will be watching this experiment with great interest (and hoping it succeeds).