[mike_hearn]

Kinda weird to ask when we're at the dawn of AI. There's relatively low hanging fruit everywhere right now thanks to LLMs, the industry will be digesting use cases of just the tech we have today for the next decade or more. It's not even that hard to get caught up with the research and start kicking out research ideas. Two weeks ago I was telling my father that the next step in transformer efficiency would be an algorithm that worked more like diffusion models, in which many tokens are predicted simultaneously with the model incrementally pushing them towards the correct answer. Just a day or two ago, CLLMs are announced and do exactly that. My maths isn't good enough to write that sort of paper but the idea was so low hanging that even a dabbler could work out what to do.

And that's ignoring all the bits of tech we use every day that could just be a lot better. JVM build systems are a mess, there is a serious need for competition there, to pick just one example that bothers me regularly.

Once you get into the swing of it the problem is not gonna be finding ideas. The problem will be finding ideas that other people will pay for. That's the hardest constraint.