[johnny_snq]

We are going to give real life scenarios based on the various roles you are testing with.(basicly a ssh login to the test machine) And we plan to asses the candidates way of solving the problems based on what commands and what process it uses to identify and solve the problem.

[globalgobble]

Please allow me my honesty but that's a stupid way of thinking about candidates and the roles. Or I do not get it. So you are basically going to judge the candidate based on what criteria ? If they use vi instead of cat to append to a file ? So decide what is the best way of solving the problem ? Or you give +1pt for using tail instead of opening the whole file in VI ? Are you aware of the fact that there are many types of systems and products that sysadmins use ? They do not behave the same. It makes no sense to use "top" on Solaris even the number for "uptime" reported by Solaris are different. What about Windows sysadmins ? What about stuff like Veritas ? What about docker ?

[johnny_snq]

We are aware that there are more ways to skin a cat. I think it's fair to evaluate a candidate based on wether if he can solve an issue or not and the amount of time it took. I agree it would be stupid to score based on individual commands and we didn't plan to go that low level. But we plan to count the number of commands with typos or wrong parameters. Don't you agree this would be a way to differentiate between 2 people that solved the same problems in the same amount of time?

[globalgobble]

Sorry I have to disagree here as well. I do not think the time or number of commands or typos should be measured. I do not think you can do any quantification here. I believe a lot of sysadmins (talking about unix) do w;uptime;inspect syslog as soon as they connect to the system. Most of the time it's just not necessary to do this but for a lot of sysadmins it's a habit that actually saved their asses several times.

Does it make sense for a sysadmin to know a numeric notation for chmod ? Probably. Would I ask that on an interview ? Nope If you need to ask why then you have probably never worked in a heterogeneous environment.

[languagewars]

When I used to work in OS support my actions were optimized for the assumptions of a default minimal install and being able to describe actions to a person of questionable knowledge over the phone and interpret what they would most likely try to read back at me. The certifications were along those lines too.. While the exact nature of the certifications was ridiculous and would be even more ridiculous in terms of judging a general user's skills, dropping an administrator on minimal installs with broken ttys, etc makes a lot of sense. If you can't recover a system when vi is not an option then you have a problem, if you spend a lot of time recovering a good term when you could have used tail, then you are also not ideal.

The only suggestion I would make is that a good tool is present and working then I don't expect a penalty. I expect tools to not be present accordance with standard minimalist setups and to pay extreme penalties if I need to call a package manager or move to a less minimalist install to complete the task.

[globalgobble]

I strongly disagree. We should not test edge cases and specific scenarios. We are trying to test the candidate and his abilities in daily tasks. If I say I use vi instead of "commands" I will probably solve most of the "text" editing tasks faster than an admin that only knows how to get around using commands.

I can come up with tons of scenarios when where you do not really know the internals you can get stuck easily. Like if I do chmod -x on the chmod binary. Does it prove anything if the candidate is not able to solve such task ?

[CmdrSprinkles]

While I mostly agree with you, I can speak from experience with how we handled the first point:

We were interviewing a new admin for the sysadmin/maintenance team and her answers to "how would you solve this problem?" were "Open that file and check that these variables are set and, if not, set them". One of the more jerkish people there went down the rabbit hole of "How are you going to edit that file? What if you don't have vim? Okay, what if you don't have emacs? Nano is gone too" leading to the response that made us hire her: "Well, in that case the system is probably completely hosed and we need to stop configuring things and focus on recovery. But I think you are looking for me to tell you a sequence of commands that involve cat. If you give me five minutes I can check stack overflow and get back to you with those"

Which I personally think should be the response for stuff like that. Understand there is an alternative and a way to resolve the corner cases and know how to get that info if needed.

[globalgobble]

Brilliant answer :-). I totally agree with you. But this is the problem..you cannot implement such a thing to a tool like sysadminarena.

Actually I believe being a sysops/devops or whatever you want to call it is one of the "most" challenging jobs in the IT industry. You get zillions of technologies you have to administer and maintain but it's impossible to master all of them. What makes a good admin is the ability to learn,adapt,ask for a help and knowing where to look for an answer. It's easy to look up the parameters if you know the fundamentals.

Interviewing for me as a devops person is one of the most frustrating experiences. You are often asked questions that are specific to the environment of a company you are interviewing with. It's unbelievable how many times I've been asked about parameters of tar, du etc. It's totally stupid. And imagine interviewing with a company that uses only a specific subset of technologies and they heavily emphasize on that.

[CmdrSprinkles]

I would argue that basically any tech job (and probably other fields) is about knowing how to solve a problem (I really do wonder what the medical doctor equivalent of stack overflow is).

And just targeting a limited subset is kind of reasonable. Because, odds are, you will be. If you are running infrastructure for a web server you probably will have decided on a few solutions and have time to experiment with new ones on a new platform. Scientific computing? Same deal. It is the same logic by which coders should learn a range of languages and tools but will probably use a very small subset at any given job.

And I think that the semi-automated tools COULD detect stuff like that. One point for each solution presented and maybe something akin to those god awful rubrics for the more complex ones. So going with the tar approach (after twenty years, I know xzvf and czvf and can use a test file to figure out the order):

1 point for a correct answer 0.5 points for typing "man tar" or "<google> tar" And then a subset of the remaining 0.5 for each letter and the order of operations. So something like tar xvf foo.tar.gz ./foo would get you most of the credit.

The big problem with stuff like this is that it should ONLY be used for an initial filter, but it will inevitably be used as "Only the top N people matter".

[languagewars]

Hmm, it seems like you are looking at how you should triage an applicant if they let you test them very painfully. I am interested in games I can play where mastery also serves as a potential initial triage and could lead to interesting discussions.

Just as I like project euler gamified as a fake ideal programming project, I would like a systems game where the point is optimizing for the worst case with minimal tools.

Add an IDE, a choice of editors, all kinds of softskills requirements that are more significant to an employer than these base primary skills. Now, I wonder why I am working for free as I am no longer just practicing our core/universal skills in an idealized/timeless environment. The better place to do that form of showing off work realistic skills is in OSS projects.

[marcosdumay]

Do you work as a sysadmin? If so, how many times did you recover from a broken tty at work? (I mean, is it a number bigger than zero?)

I've been one for a short while, and get to play devops from time to time. When I see a machine so broken that it can't give me a tty, it's time for a reboot (if last command caused it - actually don't remember it ever happening), full reinstall, or trash. I won't lose any time recovering it. If there's some important log on the disks or anything, mount the disk on another machine.

Now, I've recovered a broken machine plenty of times at home. I've edited files with sed and pipes because I broke every other editor available. I have never actually edited the memory with sed, but I could if I had to. But all that is at home, where it's fun to create a crazy configuration, and totally not fun to throw things away. It's fun, but completely unproductive.

[languagewars]

> Do you work as a sysadmin? If so, how many times did you recover from a broken tty at work? (I mean, is it a number bigger than zero?)

Before then I was an sysadmin for a little while, but I don't really see what difference the role makes. I've never had a job where I didn't at some point need to deal with a legacy serial console of some sort misrouted through incompatible aggregators, etc, in order to deal with a networking problem and/or auth problems.

The idea that I would blind reboot a system in a failure mode I can't diagnose is not something I would normally consider until I've exhausted all means and only if I know a fair bit about it. I'm sure some of my colleagues did things like that as some were developers with relatively little system knowledge, but only a few were stupid enough to mention losing failure data through laziness to senior staff.

> It's fun, but completely unproductive.

? If I had a page for you should have diagnosed and every time you hit a swiss cheese failure with that attitude than I wouldn't use computers at all anymore.

[alexgotoi]

Thank you for your input and suggestions, highly appreciated.

[throwawayosiu1]

1. How exactly are the sysadmins graded? (because in the end, let's be real...this tool will most probably be used by HR folks to filter out candidates and they'd not care too much if you send over a screencast/replay of what the candidate did)

1.1) Sys admins generally have their own set of tools (which would be customized per company I guess)...would these be installed by default? (I'm assuming you are provisioning docker containers / candidate / test and then destroying them after use).

1.2) I've used the ones that are sent for developers and it's been a pretty terrible experience at best (it's buggy, won't accept valid code etc.). If this is the case for sysadmins, it's even more worse (but then again, I guess you can put a spin on it and claim it's for the best as they'd have to deal with unpredictiability).

2. How will different technologies be integrated to test candidates on? (for example, testing knowledge on multi node kubernetes cluster and so on).

[johnny_snq]

1. I'm thinking the best grading sistem would be yes / no ? rigth ? If you manage to solve the problem and tests pass you pass. 1.1 If you get root access on a machine you have the freedom to add whatever tools you feel fine. 1.2 I don't think doing a terminal emulation is the way to go. 2. Advanced tests are on our plans but we haven't got everything figured out yet.