[Scene_Cast2]

I worked as an applied ML researcher for a while, so I'll give this a shot.

"- AI can solve any problem across modalities—just feed it data." - a large chunk of my time in ML is spent on data. I can't emphasize this enough - obtaining large amounts of quality data is a primary challenge with any sort of ML task. This might get easier with time, but will remain a challenge.

The corollary is that niche applications (and thus good fundamentals) are still important.

"- Are the challenges you encounter just a matter of “more compute/money,” or are they fundamental barriers?" - Well, there's a spectrum. Hallucinations are inherent to ML models - I don't think anybody has cracked ML model confidence estimation, and plenty have tried.

A slew of current limitations around LLMs stem from limited context windows. That is "only" inherent to the Transformer architecture (and there is some ongoing work on alternatives such as Mamba).

I think that "agents" and deep integration with computer interfaces will have some interesting automations come out of it.