[ruuda]

A slightly more recent post, that really opened my eyes to this insight (and references The Bitter Lesson) is this piece by Gwern on the scaling hypothesis: https://www.gwern.net/newsletter/2020/05#gpt-3