[jsheard]

> slow (<400G) networking

We're not getting Folding@Home style distributed training any time soon, are we.

[krasin]

Distributed training data creating & curation is more useful and feasible. Training gets cheaper 1.5x every year, but data is just as expensive, if not more, given that the era of "free web crawls of human knowledge" is over.