[philippotto]

You could make a first evaluation on the client so that the user gets quick feedback whether his solution is right. To validate that the user didn't cheat, you can make a second evaluation (server sided). The user shouldn't be forced to wait for the server sided evaluation. Instead he should be able to solve the next task immediately.

But nevertheless: Great idea with a lot of potential! Maybe you can give more detailed compile errors, if the user wrote incorrect code.

[gizzlon]

If you do implement this, you can also avoid validating every submission server-side. It should be enough to check a random sample. If you suspect someone, you can start checking them more aggressively.

If you want to get serious, there's actually a lot of research into cheating prevention :)