[brandall10]

When Attention is All You Need was published, the market as it stands didn't exist. It's like comparing the pre-Jordan NBA to post. Same game, different league.

I'd argue the top individual researchers figure into the overall AI spend. They are the people leading teams/labs and are a marketable asset in a number of ways. Extrapolate this further outward - why does Jony Ive deserve to be part of a $6B aquihire? Why does Mira Murati deserve to be leading a 5 month old company valued at $12B with only 50 employees? Neither contributed fundamental research leading to where we are today.

[jdefr89]

Seriously. The transformer coupled with tons of compute is why we got here. When that paper came out and people (AI researchers) saw the results many were confused or unconvinced. No one has any clue such an architecture would yield the results it has. AI systems has always been far more art than science and we still don’t even really know why it works. I feel like that idea being stumbled upon was sort of more luck than anything…