The whole point is that Newtonian mechanics doesn't uniquely predict the motion of the ideal ball on that ideal shape. The ball could stay there forever, but it could also start moving down along the shape at any point in time - both are valid possibilities in the idealized model. This is the unintuitive part.
The ball could stay there forever, but it could also start moving down along the shape at any point in time
Only if the fourth derivative spontaneously changes from zero to nonzero. It doesn't seem any more surprising than the conditions f(0)=f'(0)=f''(0)=0 not uniquely determining f(x) for all x.
The condition imposed by the construction of the problem and the laws of motion is that f''(t) = sqrt(t), and that f''(t) = 0 => F(t) = 0. The function given as an example in the article, f(t) = {(1/144) (t-T)^4, t >= T | 0, t < T}, obeys both laws, just as much as f(t) = 0 does.
I'm not sure what the fourth derivative has to do with this argument.