[seanmcdirmid]

I heard that Google was actually late to the game on this, not realizing the value of GPUs over (just) distributed computing when the DNN trend started. Now they even have custom silicon...

[daviddumenil]

I think it was more a case that Google had a large existing investment in CPU-based compute.

A lot of Dean's initial work was getting models effectively training on that kind of hardware using distributed SGD.