[argonaut]

While everything you've said is technically true, straight out of the textbook, I don't really see how it contradicts what he says.

These high-capacity models (neural nets, decision trees, boosting) do overfit like crazy and tend to be used as black boxes without any domain knowledge. The key in his statement is when he says "given enough data," because having tons of data is one of the best ways to combat overfitting (given enough data, variance is negligible). And the fact that we can measure how much they overfit and take steps to regularize doesn't change the fact that, for example, deep learning is really way more of an engineering discipline than a mathematical or statistical discipline. And these are not criticisms of those areas at all: those are exciting areas of research precisely because there are so many unsolved problems and areas where we are working without a solid understanding!