|
|
|
|
|
|
by kortex
1824 days ago
|
|
|
> And, in fact, quite insulting to the intelligence of even the most casual ML engineers. Exactly, casual ML engineers. The issue of plateauing tends to occur because there is no more novelty to be had in the data. What mega-experiments like GPT and similar have shown us is that actually you can keep adding novel data and keep improving the model. Kinda inelegant, yet effective. The problem is, most institutions can't add more novelty beyond a certain scale, since that usually means shoveling more money at data storage and compute, on top of the novelty collection. Tesla merely has to open the money tap to get more of both compute and storage, and let the real-time data flow in. |
|
|
And if you watch the other parts of the presentation, you'll see the bits about them buying clusters with 5k+ A100 GPUs. Presumably they intend to do something with those. Probably not streaming Fortnite concerts.