[josephg]

It can. I love using debugging problems as an interview question.

Give the candidate some simple code (~200-300 lines is probably about right) with a handful of failing unit tests. Sit them down, walk them through running the test suite and ask them to find and fix as many bugs as they can in 30 minutes.

It pays to make the first bug trivial (like, a typo or something) and make the subsequent bugs increasingly subtle. You learn an awful lot about a candidate by watching their process.

The hard thing with this sort of question is calibration. Candidates will be able to debug way less code than you think in 30 minutes. It helps to run the code past a few coworkers before giving it to candidates to figure out if its too hard a test.

The surprising thing about this test is that it seems to be really highly correlated with senior engineers. Senior engineers often don't do as well as smart grads at coding problems because we often don't write that much code any more. But senior engineers seem to do waay better than juniors at debugging problems. If I could only get a candidate to do one programming problem, I'd get them to debug code. I think you get a massive amount of signal about whether you'd want to hire someone out of a debugging challenge.

[gregmac]

Debugging works extremely well during an interview because:

1. Your evaluation can be interactive: you're watching them step through, can stop and ask questions about their thought process, etc. If it's a theoretical problem (ie: not on a live computer) you can even re-calibrate on-the-fly if they're really quick (maybe a lucky guess) at narrowing down the issue, or just asking "what if this happened instead, what would you do?"

2. Grading the candidate is subjective, just like the rest of the interview process.

The OP was talking in the context of college-style evaluation. I don't think you can apply either of these things to grading someone in a course. (1) doesn't scale, and (2) isn't a fair (or unbiased) way to evaluate students.

[josephg]

Sure you can.

I’ve done “prac tests” before when I was studying CS. The test happened in a computer lab. We were given some specs and had to submit programs which implemented the specs. (The specs were written in a way that grading could happen automatically).

Just do the same thing, except the student is given some code with some failing tests. Their grade is determined by how many of the bugs they can fix within the time allotted.