[adamsmith143]

Diminishing returns? Have you read the Gato, Palm, Stable Diffusion, etc. papers? Progress is racing ahead. Nothing is stalling... the only thing stopping progress from accelerating even faster is data.

[random314]

He is talking about Deep learning at FB

[mirker]

Many of these scaling patterns are logarithmic with respect to data size. You can only double the dataset size so many times that it’s really not clear the scaling will continue.

[lostmsu]

Low data modes are also progressing quite fast. There is Dreamer and more recent papers based on RL in learned world models.