[hedgie]

I often do ask at least one of these questions on an interview. I don’t do it because I care about a precise answer or whether you know the exact dimensions of a golf ball. I care simply because if you can’t do Fermi calculations, you can’t make long term architectural decisions. You’ll build a system which handles 2x today’s load very nicely and which I need to replace in 2-3 years, or you might overarchitect a system which can handle 1000x more load than I’ll ever need.

...why not just ask a question about building a real-time monitoring system itself and judge the responses there?

i would never pass this test. i would solve this problem using variables for dimensions of the bus and a constant "packing factor" (or assume the optimal cannonball packing and approximate the golf balls as spheres) and write an express formula for the solution. then i would substitute in various constants for the dimensions and adjust the packing factor to find a range of solutions.

the problem with the question is they want an intuitive solution. giving them this formula and evaluating for a range of variables would just piss them off, but it's the best way to approximate the answer of something that poorly defined.

[brey]

> ...why not just ask a question about building a real-time monitoring system itself and judge the responses there?

agreed. these abstractions for their own sake don't help, make the candidate feel like they're jumping through hoops for no reason

and they're intrinsically less useful than simply talking about real problems you've faced; you can't follow up an answer about golf balls down various interesting avenues of discussion when a candidate touches on some other area, because your job is not packing golf balls into a bus.

[ajdecon]

i would never pass this test. i would solve this problem using variables for dimensions of the bus and a constant "packing factor" (or assume the optimal cannonball packing and approximate the golf balls as spheres) and write an express formula for the solution. then i would substitute in various constants for the dimensions and adjust the packing factor to find a range of solutions.

Oddly enough, this was exactly how I was taught to solve Fermi-like problems in my physics program. The point was not so much to be able to "intuit" the answer as to learn how to find approximate solutions with little or no real data. You could do this sometimes just with off-the-cuff estimates (as in the piano tuner example in https://en.wikipedia.org/wiki/Fermi_problem), but often enough we came up with approximate formulas and guessed at the constants.

I haven't been in enough startup job interviews to know if actually doing algebra would make them think less of me, but that would really seem pretty dumb.

[hedgie]

that may be an unfair assumption on my part. i just assumed that when people ask these abstract riddles, they're really looking for some way of testing your intuition and not something this mundane and boring.

it really boils down to basic algebra with some constant fudge factors. meanwhile, cs people have a host of sophisticated tools to estimate asymptotic behavior for the worst, best, and average case, or error bounds on solutions to NP complete problems as a function of the number of iterations of approximation algorithms, or trade-offs when certain pre and post conditions on the data influence the run-time of canonical algorithms (think searching).

the basic toolset of computer scientists is so advanced i guess i'm shocked anyone would ask such a relatively simple problem when there are much better ones out there.

[ajdecon]

Warning: I don't actually do web startups myself, so this is based on an outsider's view of the field.

Computer science provides some really awesome ways to deal with algorithmic problems... but for most startup-type applications, those problems only come into play in a few places (if that). A cool time-management webapp might have an interesting algorithm at its core that uses machine learning to auto-decide what tasks are most important (making something ridiculous up), but the rest of the app is basically just a shiny interface to a database.

I really doubt the problems of scaling a CRUD webapp (at less than Facebook scale) lend themselves to good mathematical analysis. Even if they did, these apps involve lots of third-party layers glued together to put together interesting functionality, and those seem to change often enough that it's not worth even trying detailed analysis.

So instead they do Fermi-problem estimates to figure out their basic scaling relationships. Then you can decide how to plug things together, decide what your basic resource constraints are (compute, memory, disk, bandwidth...) and make it easy to add those resources when your customer base expands.

Edit to add: Thinking a little more, that's exactly when you'd want a set of approximate algebraic constraints, so you could decide how many customers-per-EC2-instance you could serve and how that number might change as instance count increased. Math, it's important.

[hedgie]

that's a good point.

at my large company i do this every day on the embedded system i work on, which luckily is not a CRUD app. i guess i would hope they would ask me about that XD

databases can get tricky. but you're right, if something complicated like a bloom filter is used to solve subset-query it's probably in an third-party API they grabbed somewhere that handles most of the difficulty.

[malandrew]

The last thing I would think is desirable in a job candidate is their acceptance of problems that are poorly defined and that at best have dubious benefits.

Trying to come up with an approximate answer to a question like that without at least questioning the problem itself or at least trying to narrow it down to something well defined is not a desirable trait in someone on whom you are bestowing responsibility.

I would look more highly upon a job candidate that questioned why anyone would ever want to pack a bus with golf balls to see if there is even underlying benefit to actually doing so. I would expect the interviewer to be prepared to give a pretty good business reason for how packing a ball with golf balls will increase revenue or customer satisfaction.

[hedgie]

that formula narrows it down pretty well once the constants are defined and takes about a minute to determine. with more time than is found in a typical job interview you could actually go much further.

the real criticism of the question is that the real skill they're trying to test is better determined by directly asking about Fermi calculations relating to computer software and hardware.

the golf ball question could be interesting in it's own right, but it's asked in a context where additional information to optimize the solution is not available and an optimized solution couldn't be determined due to time constraints. hence the formula sucks and you end up simplifying it with fudge factors that only demonstrate that you are able to use basic algebra to model the volume of a bus and a golf ball.

see:

http://en.wikipedia.org/wiki/Sphere_packing

there are numerous practical reasons you would want to solve this problem. for example, say you work in an orange factory. you want to pack oranges of radius n. perhaps you would like to determine and minimize the amount of shrink wrap you have to stretch around them for transport, or keep track of the materials cost for accounting purposes.

that's essentially the same problem and uses the same formula, you're just solving for the dimensions of the container instead of the number of golf balls. you can save a lot of money minimizing the wrapping material if you pump out hundreds/thousands of packed oranges a day and choose an optimal packing.

alternatively, if you have to ship it in a non-optimal packing like boxes, perhaps it's good to calculate the materials cost to pack your product to estimate future expenses.

that article lists other problems where sphere packing is integral to the solution. just because an application doesn't seem obvious doesn't mean there isn't one.

if someone honestly expected me to challenge this interview question without understanding how it could be useful i would rather not work there. people have been studying this problem for practical reasons ever since they had to stack cannonballs for transport, it's not exactly a contrived example.

[malandrew]

Golf balls on a bus are a contrived example.

I understand the value of sphere packing (but IMHO ellipsoid packing is more interesting), but why not just ask the orange packing question instead of golf balls on a bus?

A question about orange packing in an orange factory simply does not elicit the reaction of "let's see if this monkey can dance" which one would have if asked about golf balls on a bus.

An intentionally absurd question, even if it is an abstraction of a non-absurd question, does not make that question any less absurd. If you want to abstract, abstract, but do not abstract then unproductively obfuscate, because when you do, you've taken a perfectly legitimate problem and made a trick question out of it.

I don't know why, but I feel like this Calvin and Hobbes comic strip feels appropriate: http://imgur.com/gallery/P6bwL

[hedgie]

i agree completely.