[otasevic]

I'm a co-founder of Refdash. Thanks for the thoughtful comment. I largely agree with things that you said. I think that interviews need to become significantly less stressful and more objective.

One way in which we're approaching the stress of interviews is to strip away the bad outcomes and allow people to approach interviews as "there is nothing to lose". Another way is experimenting with different types of interviews such as group interviews or project based interviews. Sometimes the issue with those experiments though is that they're even harder to standardize.

I think that in addition to what you said, we need to get to a measurable level of improvement. We try to measure the repeatability of our sessions in two ways: 1) when the same session is evaluated by two different interviewers, the ratings across dimensions should be equivalent and 2) (this one is harder to experiment control) when the same person does two interviews, they should be evaluated equivalently on the intersection of sets of things that were evaluated. By optimizing for these two ways of repeatability, we are hopefully going to move toward much more objective driven evaluations that capture the engineer's understanding of different areas and not just the spur of the moment.

[geebee]

Thanks for your response.

Based on what I've seen from your site, this does look like an improvement.

An interviewee can prepare properly, and take the interview once prepared, rather than doing this at arbitrary times that may be very busy, simply because an interview came up.

Because the interview results can be used in multiple places, there's no need to do this repeatedly for companies that have confidence in the exam (I hope!).

There are reassurances that the interviews are conducted by experienced engineers. I'd be interested in hearing more about this.

There is actual feedback, which is critical. I interviewed at google, and my understanding is that there are actual, numerical scores in a database for my performance in the various interview exams, but I'm not allowed to know what they are. To me, that's a huge problem, and I'm glad to see from your example that you provide feedback.

[otasevic]

Yes, you get to fast-track through interviews at multiple companies.

As far as the experience of the interviewers goes, we're pretty careful about it in our on-boarding process. Even more than the experience we look for open-minded engineers who are willing to question their evaluation and constantly work on improving the overall process.

Re: feedback at Google. It's hard for companies to give you access to feedback because of potential legal repercussions and also because it is very hard for them to deal with back and forth that would inevitably occur.

That's another big point that we emphasized early on that seemed like a crazy proposition - fully transparent interview process. That is, anything that a company would see, you see beforehand.

[geebee]

It's hard for companies to give you access to feedback because of potential legal repercussions and also because it is very hard for them to deal with back and forth that would inevitably occur

Although I agree with this point, it only explains why a bad practice occurs, not why it should happen. If anything, it indicates why google might not be a good institution to be conducting exams. What you're saying here (accurately) is that because Google could be sued for how it evaluates and uses its exams, they just keep the results secret.

To drive the point home - there is currently a class action forming against google for age discrimination. However, the process is opaque enough that people probably don't really know if there has been any age discrimination. One possible data point that might be of interest would be to see the exam results, or other information about how the exams were graded (I believe there are even images of the code I wrote stored in this database for google's record keeping - but not mine. Again, I'm not allowed to see them). So, one way to discriminate and get away with it would be to simply deny anyone access to the data used to evaluate them. That way, how would they ever know?

I understand this is true of all interviews, not just tech. But tech really is unusual (perhaps unique) in that a huge part of our "interviews" truly are exams. They're not "tell me about your experience". They're "write code to find all sub matrices in an NxN matrix with a positive determinant", often at the whiteboard, in 45 minutes.

If that sounds extreme or excessively suspicious, I'd like to point out that we're talking about a company that was clearly, glaringly guilty of an industry wide collusion to suppress wages and maintain no-hire lists. I'm hoping you've read those articles and seen those emails. It's reasonable for people, at this point, to be looking for some kind of outside regulation and transparency.