I let any of my interviewees know that it is ok to use google, I just want to know what they searched for. I'm not running a trivia contest, so if they forgot the name of something or need a little refresh, that's fine.
The way I combat this is by asking questions about their current jobs, then diving deep into the technical problems they faced. It requires a lot more preparation for each candidate, but it ensures it’s not a trivia test but a discussion.
One of my favorite interviews (as an interviewer) was when a SDET candidate provided a link to their website. When I visited it there was an issue loading some page. So I asked him about it and how he would troubleshoot, then asked him to come back on Monday with a solution. On Monday, the site was fixed, and he explained to me what happened in AWS, how he figured it out, etc. So he was hired, going on 2 years now and doing very well.
I don’t need robots who can recite what a b-tree is (ChatGPT can do that). I need people who will work hard, can understand the big picture and how to approach problems, while being a good personality on the team.
My other favorite is very open-ended questions. I mostly do ops-y interviews, and my favorite question is "what happens when you type 'curl https://google.com' in a terminal and hit enter?"
The question is so broad there isn't a "correct" answer to Google, and it crosses enough domains that any article they find is going to be too long to skim. Then I ask them to really zero in on some aspect of it they feel comfortable with and give detail. What syscalls happen to start up curl? How does the OS know how to communicate with the local router? What's the entire flow to translate "google.com" to an IP?
It's also just fascinating to see which parts candidates latch on to. I had one person spend like 30 minutes just talking about TLS and PKI. Another person delved into kernel packet handling for a while.
That's what I do (instead of curl, with a browser).
MANY super cool convos spun out of this question (interviewed around 60 people). One of them never actually got to the network request part bc we went DEEP into event handlers in a GUI etc. Another candidate was all over the place with key exchange protocols and what and how can go wrong.
I usually don't ask further questions to "corner" them, let them go into any of the details they want.
Ah, I am so happy I am not the only one who invented this interview method :-)
Thanks for this. Looks like it's going deep into "client side".
Thing about "Google.com" (I used Facebook/Gmail) is that when server side stuff is of interest (as it was for this cloud engineer job), then I also want to hear about geo DNS, reverse proxies, LBs, CDNs, eventual consistency, distributed storage, etc, all the complexity that is happening once that cat video appears in the browser.
Then again we can go back to JS, CSS, JIT and others' space
I have a "greenfield" scenario I ask SRE candidates. I give them unlimited time, resources and money to build whatever systems they want to ensure that code is production ready before it goes to prod. The only constraint is that they will be the only person who ever has a pager, 24x7x365. Tell me how you make that work with high confidence that your life won't be ruined. It's not verbatim but that's the gist. So many paths to explore and I think it does a great job of leveling people.
I've only ever had to hire one person to work beneath me, but that sounds like how I ran the interview.
We were hiring students for a temp position. We're a Controls Engineering company, but my department is dealing with more traditional languages for supporting applications and needed extra help for a bigger project. I know the tech we use isn't standard in the university so the interview was asking students about how they approached their major projects and their methodology for learning new tools/languages.
The first guy who straight up said "I already know enough of C++ and Java, I suppose I'd just google how to do x in c# and branch from there" got the job. Because...yeah, that's about it. We talked about a 4th year project and what his responsibilities in the team where, problems faced, solutions found, etc.
This is how I was interviewed when I was a scientist/engineer and how I got interviewed at gov labs when I switched to programming. I still refreshed material for my interviews but they were focused on the actual job areas. I was really shocked that this was not common in the software world and feels weird that as an AI research people are asking me about leet code and not about mathematical formulas, limitations, and analysis of architectures or my own research. PhD scientist interviews (at labs and universities) are essentially a short form of people's discretions with a focus on Q&A. It's always appeared successful to me and I figured that the leet code was always because 1) momentum and 2) there's so many applications that an arbitrary filter has no realistic effect on outcomes other than reducing the number of candidates (due to the difficulties of measuring merit). (Similar to university admissions) But I think we all have to admit that meritocracy is not realistic and act under this belief. It's fine to have arbitrary filters if we recognize them as arbitrary but not if we go around and tout superiority for passing these. But I guess that's a corollary to Goodhart's Law
Because those fields are currently too limited and that would only work for researchers that are exactly in the field the company wants talent for.
Researchers' fields/interests are too narrow, by far, for companies to find enough candidates. Machine learning departments go from low thousands (FANG) to maybe a dozen individuals on the low end (small regional banks, ...). That adds up to let's say 40.000 jobs worldwide.
Plus there's the non-cheating cheating. The majority of conferences (including ICLR/NIPS/CVPR) are still presentations by companies about how they "innovated" by letting an intern use 10-year old techniques, in an established library (ie. not pytorch, but an "integrated" solution, sometimes going as far as an Oracle tool) to look at their own proprietary data (in medical, social sciences, sometimes chemistry). This then delivers them a "paper", goes into the conference proceedings, and they make sure this delivers dozens, sometimes hundreds of citations for all individuals involved.
Don't get me wrong. Delivering a major paper at those conferences is a major, incredible accomplishment that's beyond me, for example. But there's 20-30 people on a yearly basis that "really"/honestly do that and over 5000 total presentations at those conferences. And there's 10000 or more candidates needed to fill positions at companies.
And then the question is: who would you rather hire? A math phd, or frankly even a CS master with no relevant machine learning papers, or that intern?
The fact that most employers aren’t like you is keeping me from starting interviewing for a job after nearly 2 years of sabbatical. I simply refuse to participate in the typical interviewing process where I simply cannot show my skillset, 20+ years of experience and my diligence.
They are out there - when I was looking a few years back I would ask what the interview process is like. If whiteboard, I politely refused and tactfully said why. You won't be taking any FANGG positions, but there are plenty of positions out there that pay FANGG money minus the interview overhead. I've had some pretty great interviews when it turns into a conversation about a previous solution, or a project I'm working on, which have led to almost 100% offers. In contrast, I would do absolutely horrid in whiteboard or quiz-like interviews. Some people are built for it, but that's not me.
I had several bad experiences right out of school - I would regularly do programming exercises for recruiters and wasted a lot of my time I will never get back. Some of those exercises would take 3 to 5 hours, and I was told I was one of the faster candidates :/
Once I was able to build a resume and show off projects I told myself I would never go through that experience again.
I don't know if this comment will help you at all, but good luck to you! If you are in the Midwest area I know of a recruiter or two that are great people and can help get you pointed in the right direction. My email is in my profile.
Thank you, I appreciate you sharing your experience, it hits close home. I have moved back to Europe, but am actually looking at remote positions in the US. And thank you for your offer, I’ll take you up on that!
The problem is no matter how much training or systems a company has for its interviewing process, employees won’t read it and will inconsistently interview candidates (couldn’t remember List method == red flag; another interviewer rightly wouldn’t ding someone for this). I think interviewers really need to develop their critical thinking skills in how they judge talent. I’d rather have someone who wasn’t able to solve part three of an interview problem but high leveled some decent approaches that indicate, given more time, they’d be able to solve it. It’s interesting to me that we place this artificial time barrier on many interviews when in reality work is not chunked out that way and often takes quite a bit of thinking to come up with good solutions to things. On the other hand, I don’t think take home tests are a good solution for interviewing either. I think a lot of interviewers just end up copy catting whoever they trained with when watching interviews as a shadow (if they even had this opportunity).
I always gave open book exams to my students. They don't need to memorize details, but show understanding. Questions would range between "find the page in the book" to "can't answer it if you don't understand the subject matter."
Interviewing is a bit different, since there's no book, but when we looked for a junior dev, the applicants got a laptop with VS Code, the common browsers, internet access and an empty (macOS guest) account, and a few (short!) tasks to show if they understood the questions and were able to start formulating an answer. We also looked for a personality match (we're a small company) and signs of general intelligence. Worked out reasonably well.
I really dislike the more extreme approaches to interviewing that one sometimes comes across, like leet code scores. That's just too disconnected from the actual work. Unless you need your employee to grind leet code questions, of course.
I expect my interviewees to use Google and am disappointed if they don't. It's more important to me that they know how to find the right answer than it is that they just know it off the top of their head. 90% of their time is going to be spent figuring out how to solve all the known-unknowns they're going to come across.
Yup! If I see them struggling to answer a question or admit to not knowing something I will ask them "well, how would you find the answer?", to which they always half-jokingly answer "I'd Google it". That's when I'll let them know that I want to see some of that too!
Because people administering tests always have weird or arbitrary assumptions and restrictions. Part of getting through school is jumping through all the contrived hoops they place before you, even when it'd be much easier to just walk around the hoops. Job interviews are no different; different interviewers have different expectations, and many don't want you consulting outside information sources.
Ditto, 20 years on and I research and read articles when I start any new project because in one year there can be entire paradigm shifts in tooling and best practices.
yeah same. it's not a recitation / memorization challenge, it's an honest "if you ran into X at work, show me what would you do" exercise. It's hard to imagine an answer that doesn't start with "I'd probably google it." How do you evaluate the quality of the search results? How would you scan a page to quickly find the salient details? How would you test what you'd found to determine whether it was an effective solution? etc.
One of my favorite interviews (as an interviewer) was when a SDET candidate provided a link to their website. When I visited it there was an issue loading some page. So I asked him about it and how he would troubleshoot, then asked him to come back on Monday with a solution. On Monday, the site was fixed, and he explained to me what happened in AWS, how he figured it out, etc. So he was hired, going on 2 years now and doing very well.
I don’t need robots who can recite what a b-tree is (ChatGPT can do that). I need people who will work hard, can understand the big picture and how to approach problems, while being a good personality on the team.