[aleph_minus_one]

> - If you'd specialized in neural networks before the transformer paper, you'd be one of the most sought-after specialists right now

> - If you'd started making mobile games when the iPhone was released, you could have built the first Candy Crush

I disagree:

Concerning the first point: how neural networks are today is very different from how they were in former days. So, the knowledge of neural networks from the past does only very partially transfer to modern neural networks, and clearly does not make you a very sought-after specialist right now.

Concerning your second point: the success of mobile games is very marketing-centric. While it is plausible that being early in mobile games development when the iPhone was released might have opened doors in the game industry for you, I seriously doubt whether having this skill would have made you rich.

[imtringued]

There's also the problem that there is currently a huge transformer monoculture in the AI space. Everything gets better if you throw transformers at the problem.

If you had worked on anything other than a transformer based architecture post 2016, such as Mamba or RWKV, you would have wasted your time.

Mamba 3 is the third iteration and somehow I doubt that it will catch on.

[erikerikson]

I can confirm your response to the point you can first