[qnleigh]

I think there have been advancements in RL that deserve to be on this list. AlphaFold at the very least; I hope you had the pleasure of reading the reactions of researchers who work on protein structure determination and how stunned they were that their field of research had been in large part solved.

Here's a few more that come to mind:

- Work decoding brain signals that allowed paralyzed people to communicate via a computer (this all happened pre Neuralink)

- I think self-driving cars fit your criteria, even if the jury is still out as to whether this has/will soon be 'solved.'

- In gaming, Unreal Engine's Nanite is very impressive and a real technological innovation

- There are efforts to get LLMs to learn via RL and not just from training on huge text corpuses. Actually mathematics is a prime example where you might dream of getting superhuman performance this way. People are at least trying to do this.

Just because there's a lot of BS which tends to drown everything else out, doesn't mean there isn't real progress too.

[_5tsv]

I suppose with a blanket statement like "nothing is happening" I'm practically begging to be contradicted :)

Some of this stuff looks very cool. Have we started getting RL to work generally? I remember my last impression being that we were in the "promising idea but struggling to get it working in practice" phase but that was some time ago now.

[qnleigh]

Edit: after writing what follows, I realized you might have been asking about RL apply to LLMs. I don't know if anyone has made any progress there yet.

Depends on what you mean by 'generally?' It won't be able to solve all kinds of problems, since e.g. you need problems with well-defined cost functions like win probability for a board game. But AlphaGo and AlphaZero have outstripped the best Go players, whereas before this happened people didn't expect computers to surpass human play any time soon.

For AlphaFold, it has literally revolutionized the field of structural biology. Deepmind has produced a catalog of the structure of nearly every known protein (see https://www.theverge.com/2022/7/28/23280743/deepmind-alphafo..., and also https://www.nature.com/articles/d41586-020-03348-4 if you can get past the paywall). It's probably the biggest thing to happen to the field since cryoelectron microscopy and maybe since X-ray crystallography. It may be a long time before we see commercially available products from this breakthrough, but the novel pharmaceuticals are coming.

Those are the biggest success stories in RL that I can think of, where they are not just theoretical but have lead to tangible outcomes. I'm sure there are many more, as well as examples where reinforcement learning still struggles. Mathematics is still in the latter category, which is probably why Terence Tao doesn't mention it in this post. But I think these count as expanding the set of things humans can do, or could do if they ever work :)