[onion2k]

In my opinion they test the wrong things for the majority of software developers. Most people who write code don't need the ability to solve especially hard algorithmic problems. Ranking solutions to hard problems is probably useful if you're Google but there aren't many Googles out there. We like to think that we're solving hard problems (and actually, on HN, we're more likely to be), but most companies aren't. A large amount of software is literally just a CRUD form and a database.

For the overwhelming majority of software businesses it's far more useful to know if someone is disciplined enough to use good variable names than whether they can write an algorithm to count the number of edges in an acyclic graph.

[mjohn]

I've recently been thinking of using some kind of reading comprehension test during interviews. Other than poor craftsmanship, the biggest disappointments among developers I have hired are the ones who seem to struggle to quickly read and absorb information. I think having good reading comprehension skills is essential for code as well.

[planetjones]

A clean code testing tool would be far more advantageous. I understand the algorithmic challenges for a small percentage of jobs which need someone like that. However, most software development is about understanding needs, simplifying process and expressing yourself cleanly in the code. I wonder how much money is being wasted, especially in silicon valley, on bad hiring - i.e. the astronomical cost "mr complex algorithm" will cause to a codebase which would benefit from being simple.

[zby]

You are right that the biggest part of the job is easy - but testing the easy parts is useless - all programmers would pass it. A test must be hard in some way if it is to filter people.

Disclaimer: I have a stake in https://codility.com/

[jzwinck]

I assure you this is not always the case. I have worked directly with multiple people who could contrive the most exquisite solutions to really difficult problems but whose code was gibberish. Flagging such characteristics during recruitment would be valuable.

[zby]

Interesting - how would you flag it? In what way was their code gibberish?

In my experience people who understand complex concepts can also understand how to write clear code. Maybe those that you are writing about were not motivated to do that - or maybe it never occurred to them that this was needed?

Would you mind also explaining what you mean by this in "this is not always the case"? Because it does not seem to refer to my comment.

[junto]

I guess that concise implementations (e.g. in a one liner) are often a complete bitch to debug when someone else comes back to it to fix a problem years later.

It is much more difficult to debug and fix existing code than develop it from scratch. Hence, if you found the problem hard to code in the first place, then when you come to debug it again later you've just screwed your future self.

[danielbarla]

I've also seen (severe) examples of the converse - e.g. writing an 800 line monster which could better have been expressed as a regular expression (and a simple one, at that).

This leads me to think that conciseness is not necessarily as simple as we tend to think. Sure, overly concise code with clever hacks are often a nightmare to debug, but that's mostly because they are the wrong abstraction and not capturing the problem well enough.

[zby]

The problem with people writing too concise code is with their motivation not with their abilities - and so it is probably impossible to test (at least directly), because on the test they would have the motivation to do the right thing.

[talles]

I disagree.

Knowing how much to abstract and writing clear code is indeed an ability.

[st3v3r]

No. If someone has to be "motivated" to write clear code, especially given that these tests are supposed to lead to jobs, then I would have to question their abilities. Writing clear code is an ability of itself.

[logicality]

if programmers are not actually solving similar problems as part of their jobs, sites like codility provide little value over standard IQ tests. all they do is waste the candidate's time and employer's money.

[zby]

In my opinion programming is about solving programming problems. There are other important parts - like understanding user needs etc - but solving problems is at the core of it. In practice usually these programming problems are easier - but to test how a programmer is proficient in solving them you need to use harder problems or maybe do some programming speed tests (many easy problems).

[richmarr]

I think focusing on algorithms (if that's what you're suggesting) would be tunnel vision, and as a result the output of such a system will be much more weakly correlated with productivity than it should be.

Beyond code literacy, the most valuable skills for a 99% of programming roles are empathy and diligence. Unless you're working on the kind of programming that's arguably pure mathematics, algorithms are just not important.

There are plenty of ways to test diligence; empathy is a little harder to test directly but there are reasonable proxies.

Disclaimer: I'm working on a product for applicant filtering (generic rather than coder-specific) https://www.beapplied.com/

[what2Do2016]

you would be surprised how many "senior developers" I've worked with who can't write a decent CRUD app...

[akurilin]

Agreed. I'm generally much more interested in whether a developer can understand a new problem they haven't faced before, can research possible solution, can compare various trade-offs of solving them and can then implement a solution efficiently and effectively on a constrained time schedule.

[zby]

Just to make it explicit - what kind of trade-offs do you have on mind?

I have some stake in https://Codility.com - I could perhaps ask them to add a new kind of tests :)

[jk563]

This is how I prefer to use it as a tool. Rather than set up difficult questions and test some algorithm that can be found with a decent google, set up a small number of relatively straightforward challenges.

This then filters out anyone who just can't code, and you can review the passing tests for craftsmanship.

[gedrap]

>> We like to think that we're solving hard problems (and actually, on HN, we're more likely to be), but most companies aren't. A large amount of software is literally just a CRUD form and a database.

I think it's simply 'well, everyone is doing it so...'. So for some people it feels just safer to do what a lot of others do, while others can't be bothered, and a just don't know any better. I'd believe that's a very common reason for this approach :)

[hellweaver666]

I agree, where I work we use HackerRank (for most roles that require any coding, even designers) and we try very hard to set a challenge that is representative of typical daily work.

[st3v3r]

Agreed. If you're not doing that kind of stuff, then anyone you get from these tools is probably going to be bored out of their mind, and leave soon. Either that, or you're going to pay them an assload of money, and they're going to spend lots of time on "clever" things.