[bane]

Yes, I have some technical chops - started coding on a TRS-80 Color Computer II in the 80s and never stopped. It's not something I do much of anymore (sadly) as a growing crop of grey hair has convinced my day job c-levels to shovel me into managing groups of technical people.

I also do lots of work outside of pure development in my day job, and it has wildly different sets of requirements, but the process is more of less the same. I've hired and managed a few more of these types than straight developers. But I've managed to have about the same success with both, or if we did hire duds for some reason (CFOs cousin or whatever), I noted the potential problems in their file and came out reasonably accurate.

The problem with trying to hire the next legend in the field is that more often than not, you are not hiring the next legend in the field no matter how they feel about themselves.

That's why legends are notable, they're unique and rare. If they are really that awesome, they'll become legends in their own time and you'll know about them before their resume hits your inbox. Building a hiring process for legends is fraught with peril. Legends are almost by definition not hire-able around repeatable hiring processes. So companies fake a process. Some companies select for specific schools -- like the too often true stereotype of top finance firms only hiring Harvard/Yale grads or top tech firms only hiring Stanford/MIT types. I've even heard of some hiring with a heavy weight on SAT scores or average KLOC per month!

As much as we all like to think of the lone swashbuckler programmer, coding his way to greatness in the fewest undocumented KLOCs possible, I'm hiring people to build actual things that other programmers of various talent levels from other organizations might have to work on months or years later long after my rockstar has moved on to greener pastures because we didn't offer him the office chair he wanted or whatever. In other words, I'm hiring for the best of typical. I need engineers, not crusading nights. I'm building the Golden Gate Bridge or the Empire State Building, not the Pioneer Space probe or filming and episode of MacGuyver. I've hired the rockstars before, there's a reason organizations tend to eventually move on to other processes.

I've worked with some absolutely unknown solid developers who I'd put up against the best in the field any day of the week, because they produced miracles on time and within budget. Absolutely stunning, elegant, rock solid software. But I'd rather focus on their steady solid performance over time. Rather than have a rockstar come in and dump down a few thousand lines of impenetrable code then flame out. I've found that solid, steady developers will ultimately outperform those guys every time. They'll do it right, and the bridge won't fall down or the building won't fall over.

If you are looking at tremendous number of resumes per week, you have to build a hiring process that selects for the best of typical. If you spot a super star in the pile, by all means talk to them, but you'll eventually find that most of the time they have overinflated egos. Hiring people is almost exactly like American Idol. 90% of all the candidates you see are bad, often spectacularly, often embarrassingly. The worst suffer from some of the most intense Dunning-Kruger effects I've ever seen. But you might find an honest to gosh amazing developer every once in a while and it's worth it to hire them. But don't build a process trying to find them, they'll find you.

If you don't build this kind of selective process, you'll spend literally all of your waking hours interviewing people. If you like this kind of thing, and your company is willing to let you do this, awesome. I have other things to do.

An all day technical interview sounds great, but now do this with 30+ candidates. Bye bye month of November. You need to weed people out and get it down to a manageable pool that you can then focus your time on.

I'm know what I'm saying may sound harsh or overly selective based on some arbitrary metric but it's all there for very good, repeatable reasons. But in practice these are only guidelines. If you went to school, awesome for you, I don't care where, be prepared to talk intelligently about the experience, be proud that you demonstrated the ability to complete a grueling, often mindless multi-year project on your own. If you didn't, but managed to autodidact your way to a great skill set, fantastic, be prepared to show and talk about why that worked better for you than going to school.

[tptacek]

Two things.

First, I wish you'd stop caricaturing everyone hired through a process other than yours as "lone swashbuckling MacGuyvers" or whatnot. I am not laying out a method for hiring "the next legend".

Second, it is incumbent on you to get better and more efficient at hiring people, and if you're stuck with "resume, phone screen, all-day interview", you've got your hands tied behind your back. There are so many ways to control and accelerate a recruiting funnel that it's tragic to be running one for a high-tech company the same way that Northern Trust runs them for junior bankers.

[bane]

Well, not that I was talking to you (I was talking to cookiecaper) But I appreciate the interest even if your reply is a little bit of a response to something I didn't say.

Near as I can tell your process appears to be first come first serve.

By your own statements, you don't look at resumes, and you apparently don't ask any particular questions about work history or education during your interview process. I'm guessing you have some sort of technical screen since you've alluded to having some sort of highly specialized selection process of some sort.

But I've illuminated you enough with what I've learned to be a solid and reliable process, and I know I must be unfairly characterizing your process above or you wouldn't be in business, so what's yours?

[irahul]

> By your own statements, you don't look at resumes, and you apparently don't ask any particular questions about work history or education during your interview process.

This thread is already a bit too long, and I haven't read through tptacek's link about his hiring process, but:

1. you don't look at resumes,

If I am hiring for a ruby on rails position specifically, the only thing I need to check is if you know ruby on rails. I know RoR, I know my requirements, I can evaluate if you are suited to this current role.

If I am hiring for RoR, but I am looking for a generalist - someone who might not know RoR but can pick it up on the run, then looking up at the resume for RoR isn't required. I will just talk general algorithms, design problems, code snippets.

2. and you apparently don't ask any particular questions about work history or education during your interview

Me: Let's say we need to have a threaded discussion forum. You know, like HN. Where replies can have replies.

Joe: Yes, I got that.

Me: Let's see how you go about deciding upon the models for that. I need upvote/downvote/recall vote on all posts. Here, take this piece of blank paper, and run me through how are you storing posts, and comments. And when loading a particular post, how are you going to load the posts and comments associated with them.

Joe: works through it

Me: That looks great, but I am concerned about the number of db hits.

Joe: suggests some kv store suitable for it, talks about storing graphs

Me: I think that's good enough. How about implementing upvote/downvote/recall vote.

Joe: Need to change the db models. I skipped them in the first iteration.

Me: Let's skip it. You know about closures? Care to quote me a particular example of closures?

Joe: function translator(el) { return function(text) { $(el).text(text); } }

    // Translate all i18n elements
    $('.i18n').each(function() { 
        // Text and callback
        AJAX_API.translate($(this).text(), translator(this));
    }

And so on and on.

I don't see why I need to specifically ask about your education. I might ask about your work experience, but not necessarily.

[bane]

Matasano's publicly stated process is basically to call them up on the phone, then begin a multi-week technical interview process consisting of several technical screens and some project work.

It's a first come first serve, but only if you are this tall, process.

It's intriguing, and the technical interview process isn't entirely unlike others I've seen, and does a good job weeding out persons that have no business working for Matasano.

Like anything there is an upside and a downside:

Downside first (because I'd like to end on a positive note)

As an applicant, you really want to have that job, because you just dropped applying for anything else for a few weeks while you run this gauntlet.

Good side:

Matasano ends up with qualified, eager employees who really want to be there.

The biggest problem for me in both of your examples is that, taken literally, you know only the functional capabilities of the candidate, but virtually nothing else.

People who can answer moderately deep technical questions are a dime-a-dozen in certain parts of the country. So while passing an intensive technical screen will generally yield solid employees, it can sometimes fail in interesting ways.

I remember a guy my friend hired just this year who passed all of the technical screens (all done remotely via phone interviews, webex and email) with flying colors, but had two problems: once employed, he just froze unless given explicit direction every 5 minutes, he had atrocious personal hygiene that was so bad it sent half a dozen people home ill. On his resume? Didn't finish his schooling, <1 year work histories at a half dozen companies plus some other warning signs I can't go into. An in-person non-technical interview would have revealed his other issues.

Unanswered questions for the Matasano process:

1) Can they actually finish an extended task or are they just good test takers?

2) Are they self motivated?

3) Do they tend to move about employers very quickly? (ladder jumpers, flaky personalities)

4) Have they performed well on teams?

5) Do they have a violent criminal history or dangerous psychological issues you should be aware of?

6) Do they have other skills you might be able to take advantage of in the future?

etc.

Hiring people is hard. Hiring good people doubly so. But a good hiring process should be well rounded so that you understand where you can use a person in your organization if you choose to hire them, and set expectations of performance up front. An anti-social super tech type might only be usable in development, while somebody with good presentation skills and strong technical skills might be more valuable in a variety of roles.

I'm not saying mine is the best and greatest in the world, but it's pretty good, and more or less what you'll find just about any place. Some places have oddball or one-off hiring systems that work well for them and provide them with their own desired metrics on an employee. I think that's great also.

Going back to the original point that drug us down into this conversational morass, resumes are fairly standard, and have certain things most companies expect to see. Building a resume based on the hiring practices of just one company is not something I'd particularly recommend unless you only care about getting hired at that one place and nowhere else.

Important! The resume won't get you hired, but it will get you an interview. Otherwise you need some selection mechanism to determine whom to interview or you'll spend every waking moment in interview hell.

[tptacek]

We do not have candidates perform "project work".

The amount of time people spend on our tech challenges is (a) calibrated to be less than the amount of time one spends on an all-day tech interview and (b) designed to be broken up into three short chunks (by short we mean "an hour or so") so people can do them at their convenience, rather than having to take a personal day from work to "run a gauntlet".

We go way out of our way to respect people's time; in fact, after interviewing candidates from our earlier process, we designed this process with that as a primary goal. Which is why we say that over and over again on the Careers page.

"Drop everything for a few weeks" is disingenuous hyperbole. As is the idea that --- working in information security for Fortune 500 companies --- are hiring violent felons. I understand that you're irritated with how I've responded to your comments and I don't blame you for being uncivil, but please constrain your irritation at me, not my company.

[bane]

I apologize if I mischaracterized your company's hiring process -- there was no disparagement intended. On the contrary I believe it is intriguing, and given your company's excellent reputation it appears to work well.

It also appears that we're definitely at a communications impasse. By "project work" I was referring to steps 3, 4 and 5 from the career page you referenced:

3. Web app challenge

4. Custom protocol challenge

5. Write a fuzzer (including automated testing)

I'm open to any other descriptor for those steps that you care to use. I most sincerely apologize if "project" implies too heavy a meaning.

And by length of time I was referring to the answer to question #1 directly below those steps

"This looks complicated. How long does it take?

Not that long! We work hard to minimize the amount of time we demand from candidates. We'll be 80% of the way through before you're ever asked to come on-site. We can usually wrap things up inside a few weeks. "

I personally think it sounds like your process is designed to optimize around keeping the technical screening as short as possible while still maximizing the information you retrieve -- which I think is great. It's certainly a much better use of everybody's time than say, Google's multiple all-day committee interview/quiz show extravaganzas are. If I misunderstood the FAQ and the process does not, in fact, take a few weeks, I apologize.

I'm not sure if it's you or I that's going out of our way to be obtuse regarding best hiring practices, but I apologize if I brought any offense or disparagement to you or your company. It wasn't intended.

[tptacek]

Our process is obviously not "first come first serve" (we would not be "very good" at screening candidates if it was), but we're pretty deeply nested in this thread to get into details.

A good starting point is http://www.matasano.com/careers.

But: that isn't the entirety of our recruiting process; it's just the portion most visible to applicants.

[cookiecaper]

I'm not saying that you should hire people without education in hopes that you're randomly selecting a future legend, I merely intend to point out that the policy of shredding every resume that doesn't have an Education section is a very wide buckshot that doesn't actually help much of anything. You should evaluate candidates on real, relevant criteria, not arbitrary "lines in the sand".