[drooby]

Lol nice.

I’m not entirely sure what the authors argument is.

If an intelligence explosion occurs with humans and AI working in collaboration, it is still an intelligence explosion. No?

Computers will make themselves smarter - with the help of humans. They already are. RLHF is a perfect example of this.

They’re arguing that you can never take the H out of RLHF? I don’t see why not. We are physical systems just like AI. Our general search algorithm is to find systems that maximize accurate predictions. Minimize surprise. This can be coded.