[johnparkerg]

They can say that they only hire the "top 3%" because their application funnels only allow 3% of the people that apply to work for them. This has a two important implications, 1)They are the "top" by their standards and 2) They are the "top 3%" of the freelancers that apply to the site. I am sure that the real top 3% are busy dealing with job offers and referrals.

[bmir-alum-007]

"The problem is, if you're not a hacker, you can't tell who the good hackers are." http://www.paulgraham.com/gh.html

Relying on coding challenges or automatically-generated code metrics is deceptive because it tests if people can solve coding challenges, not solve real production issues or work on production apps, quickly and/or produce maintainable code. (It's about as useful as asking "why are manhole covers round" frequently asked questions interview trivia questions as proxy signals for intelligence.)

So, unfortunately, finding top coding talent is fundamentally an irreducible problem because it takes a little time & money to examine if someone is a badass coder by other badass/es reviewing some actual work and/or working with them on side-project/s.

[spitfire]

http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

100 years of industrial psychology. See page three.

   -  General mental ability (Are they smart)

   - Work sample test (of actual work they might do, NOT HAZING/Quizes).

   - Integrity. (Varipus tests available for this)

That gets you > 60% hit rate.

Things that do not work.

   - Reference checks (r 0.26) 

   - experience (years) (r 0.18)

   - Years of education (r .10)

[bmir-alum-007]

Yup. Business theater is usually stuck in maintaining in-group approval, "industry standards" and historical continuity.

My favorite HR signal which doesn't seem to qualitatively work:

- non-rigorous certifications

The best interview styles are not labeled as such explicitly, whenever possible, because it makes for artificially-awkward pressure, alters expectations and ups the hype and hazing.

[acbellini]

Toptal does take time and spend money on reviewing candidates: there are no round-manhole-cover questions, there are algorithms for which one can prepare, there is a live coding session and a project that is evaluated by an experienced developer, together with the candidate. As you say, there are badass developers evaluating other developers reviewing their work and seeing them work :)

Disclaimer: I work as a Director of Engineering at Toptal and I've held a few hundreds of those live interviews myself.

[rattray]

I'm just confused why acbellini is being downvoted here. Yes, people from a company are coming to defend their record... did she say anything wrong/misleading/meanhearted/poorly phrased?

Disclaimer: she interviewed me. (The experience, by the way, was almost identical to the one I had interviewing with AirBnB recently).

[ghufran_syed]

Joel Spolsky's article, "Everyone thinks they're hiring the top 1%" seems relevant here.

http://www.joelonsoftware.com/items/2005/01/27.html

[bbeneschott]

You're correct in that the "top 3%" represents the top 3% of the developers who actually applied to Toptal (about 11,000 from all over the world in the last couple months).

However, it's also important to know that the 3% was not arbitrary. We didn’t start by saying, “wow, we should set the acceptance rate at 3%, because that sounds nice” or we somehow knew that the bottom 97% were bad. 3% (and every value you see on our screening process) is simply the result of a numerator and a denominator (successful_applicants over all_applicants). We designed the process to be the bare minimum criteria for us to be extremely confident that we are admitting someone into the network who will thrive here.

[victorhn]

what are the technologies more in demand that at the same time also would give more chances to get into toptal?

[xsolid]

The technology stack you are using is not going to affect the chance of getting into the network. Once you have gone through the screening process, you're will be matched by a team of engineers to prospective clients based on your tech stack, availability, time zone and other factors as well.

When it comes to technologies being in demand, it really ranges all across the board: from the standard PHP, Ruby, Javascript etc, over more bleeding edge like React, React Native to niche things like OpenCV are all there.

Full disclosure: I am currently employed as Head of Community for EMEA @ Toptal