Well, it is trivial with two balls, once you get it, but it's tricky with respect to "just binary search".