[btown]

"Tell us a story about how you took down production."

A reasonable question. But if you really need to reject the candidate (for instance, if you're attempting to resist, or provide cover for resisting, someone brought in e.g. by the bad kind of nepotism), no matter how much they talk about how it led them to build systems to prevent similar disasters, as an interviewer you can selectively choose to focus on the mindset that led them to break the system in the first place.

In general, though, if you're looking for ways to disqualify a candidate based on other than merit, you should examine your own biases against the candidate. Don't be evil.

[stcredzero]

"Tell us a story about how you took down production."

We had a real need for batch queries, so I built a declarative batch query mechanism for our in-house ORM. To do this, we needed to make actual objects representing database relations. Unfortunately, the "collections of lambdas" we were using as an ORM had started getting polluted with buisness logic. So I used a parser/rewrite engine to match the un-polluted DB transformation lambdas and to substitute DB column and relation objects implementing the same logic in the un-polluted lambdas.

I checked in a massive change affecting 2500 of the ~5000 lambdas, syntactically matching the logic in all 2500 of those with the parser framework, and then also vetted everything by hand. I was 100% sure the logic was duplicated exactly. This would allow us to introduce the objects, and it would also stop the march of business logic into the DB framework. Everything checked out in QA, so we moved it to production.

Production immediately came to a screeching halt. However, in under an hour, I discovered that a coworker had polluted an even lower level of the ORM framework with business logic. (The code that called the lambdas in the first place.) We replicated that logic in the new framework, and everything worked 100%. I also came out a huge hero in the end, because the batch query mechanism I introduced reduced the average number of queries for opening a portfolio from ~2500 to only 50, and because it was declarative, it could be reused by any programmer very easily.

[DoofusOfDeath]

> Production immediately came to a screeching halt. However, in under an hour, I discovered that a coworker had polluted an even lower level of the ORM framework with business logic.

"I see. So you failed to communicate effectively with a coworker, which resulted in an hour of downtime. I didn't hear you mention anything about seeing this as a problem, let alone addressing it. Thanks for your time."

[stcredzero]

So you failed to communicate effectively with a coworker, which resulted in an hour of downtime.

Nope. He was privy to our stopping the flow of business logic into the DB/ORM code. He failed to realize the implications and communicate with us.

I didn't hear you mention anything about seeing this as a problem, let alone addressing it.

Pure projection. Of course that's a problem. Thanks for disqualifying yourself up front.

[ryanchoi]

I figure you wanted to defend your take regardless, but the quotes indicated to me that the parent was posing as the company that wanted to reject the candidate no matter what the answer was. :)

[stcredzero]

And I was roleplaying as myself, rejecting the company.

[hluska]

That was nicely done and I agree - I'd rather be unemployed than work for a company that would think through something like that! Great job. :)

[DoofusOfDeath]

That's right. I was trying to play devil's advocate by showing that a sufficiently malicious interviewer can throw mud on almost anything.

[stcredzero]

I was trying to play devil's advocate by showing that a sufficiently malicious interviewer can throw mud on almost anything.

Or try too hard to, and reveal their true intentions.