https://epoch.ai/blog/can-ai-scaling-continue-through-2030
There is plenty of data left, we don’t just train with crawled text data. Power constraints may turn out to be the real bottleneck but we’re like 4 orders of magnitude away