|
I've been on both sides of HackerRank. I helped develop/configure a HackerRank test for candidates for my team, and recently I've taken a HackerRank test while applying for a position at another company. HackerRank is just a tool. Its effectiveness depends on how well the company interviewing candidates configures it. I think algorithmic questions are the most popular, but it's completely configurable; you can have it ask whatever you want. It's also possible to manually review submissions. When we used it to evaluate candidates, we'd manually review the code for candidates that scored somewhere in the middle of the range. Depending on what their submission looked like, we'd decide whether or not to proceed with them. (HackerRank lets you see each version of the code attempted by the user, in addition to the final solution submitted.) We actually found it particularly efficient at finding good candidates; there was a very high correlation between interview performance and HackerRank score. If properly configured, HackerRank makes it easier to identify good candidates, which is (IMO) a good thing for everyone. For companies, it means that they spend less time interviewing bad candidates, and for candidates themselves, it means that they might be able to get their foot in the door somewhere where they'd usually get blocked by the "resume scanner" filter (since the company isn't risking engineer time/productivity to send out a simple HackerRank evaluation). That said, HackerRank isn't perfect. My biggest complaint is the lack of feedback for some failure modes, most notably segfaults and failing non-sample testcases. For segfaults, it simply returns "segmentation fault" and you're expected to be able to find the problem (a similar tool I've seen, coderpad.io, dumps a stack trace). In some algorithmic questions, non-sample testcases include data that is vastly more voluminous than the sample data (which is intended to catch non-optimal implementations of the algorithm), but this isn't obvious at all. It would be nice if the non-sample test cases had titles (e.g. "extremely large input" or "edge case") so you could theorize about why yours failed. People who have experience using HackerRank have a definite edge over candidates who have never used it before. If you are planning on taking a HackerRank test for a position, I would recommend trying some open questions on their site first. I also recommend having a local text editor, compiler, and debugger ready in case you hit a segfault that isn't immediately obvious. If your solution fails with "Terminated due to timeout", or your code works on all the sample cases but fails/crashes on the hidden cases, then your algorithm is likely not efficient enough (in the timeout case, look for ways to speed it up; in the 'mystery crash' case, look for ways to reduce memory usage). Lastly, if you have extra time after completing a HackerRank test, I recommend making sure your code is as clean as possible and is well-documented (but not over-documented), in case they decide to manually review it. |
* the stack trace -- We avoid showing stacktraces to candidates as it has the potential of leaking information (a well created error message/stack can leak test case data).
We can and should show the trace (for languages that provide it) to reviewers. I've added an item to our internal tracker.
* Familiarity of environment: Most (if not all) of the challenges companies created by companies link to a sample test (hackerrank.com/tests/sample). We notice many but not all candidates take this before the actual challenge. Perhaps we should highlight this more.