[globalgobble]

I strongly disagree. We should not test edge cases and specific scenarios. We are trying to test the candidate and his abilities in daily tasks. If I say I use vi instead of "commands" I will probably solve most of the "text" editing tasks faster than an admin that only knows how to get around using commands.

I can come up with tons of scenarios when where you do not really know the internals you can get stuck easily. Like if I do chmod -x on the chmod binary. Does it prove anything if the candidate is not able to solve such task ?

[CmdrSprinkles]

While I mostly agree with you, I can speak from experience with how we handled the first point:

We were interviewing a new admin for the sysadmin/maintenance team and her answers to "how would you solve this problem?" were "Open that file and check that these variables are set and, if not, set them". One of the more jerkish people there went down the rabbit hole of "How are you going to edit that file? What if you don't have vim? Okay, what if you don't have emacs? Nano is gone too" leading to the response that made us hire her: "Well, in that case the system is probably completely hosed and we need to stop configuring things and focus on recovery. But I think you are looking for me to tell you a sequence of commands that involve cat. If you give me five minutes I can check stack overflow and get back to you with those"

Which I personally think should be the response for stuff like that. Understand there is an alternative and a way to resolve the corner cases and know how to get that info if needed.

[globalgobble]

Brilliant answer :-). I totally agree with you. But this is the problem..you cannot implement such a thing to a tool like sysadminarena.

Actually I believe being a sysops/devops or whatever you want to call it is one of the "most" challenging jobs in the IT industry. You get zillions of technologies you have to administer and maintain but it's impossible to master all of them. What makes a good admin is the ability to learn,adapt,ask for a help and knowing where to look for an answer. It's easy to look up the parameters if you know the fundamentals.

Interviewing for me as a devops person is one of the most frustrating experiences. You are often asked questions that are specific to the environment of a company you are interviewing with. It's unbelievable how many times I've been asked about parameters of tar, du etc. It's totally stupid. And imagine interviewing with a company that uses only a specific subset of technologies and they heavily emphasize on that.

[CmdrSprinkles]

I would argue that basically any tech job (and probably other fields) is about knowing how to solve a problem (I really do wonder what the medical doctor equivalent of stack overflow is).

And just targeting a limited subset is kind of reasonable. Because, odds are, you will be. If you are running infrastructure for a web server you probably will have decided on a few solutions and have time to experiment with new ones on a new platform. Scientific computing? Same deal. It is the same logic by which coders should learn a range of languages and tools but will probably use a very small subset at any given job.

And I think that the semi-automated tools COULD detect stuff like that. One point for each solution presented and maybe something akin to those god awful rubrics for the more complex ones. So going with the tar approach (after twenty years, I know xzvf and czvf and can use a test file to figure out the order):

1 point for a correct answer 0.5 points for typing "man tar" or "<google> tar" And then a subset of the remaining 0.5 for each letter and the order of operations. So something like tar xvf foo.tar.gz ./foo would get you most of the credit.

The big problem with stuff like this is that it should ONLY be used for an initial filter, but it will inevitably be used as "Only the top N people matter".

[languagewars]

Hmm, it seems like you are looking at how you should triage an applicant if they let you test them very painfully. I am interested in games I can play where mastery also serves as a potential initial triage and could lead to interesting discussions.

Just as I like project euler gamified as a fake ideal programming project, I would like a systems game where the point is optimizing for the worst case with minimal tools.

Add an IDE, a choice of editors, all kinds of softskills requirements that are more significant to an employer than these base primary skills. Now, I wonder why I am working for free as I am no longer just practicing our core/universal skills in an idealized/timeless environment. The better place to do that form of showing off work realistic skills is in OSS projects.