[doctorpangloss]

Your agent may need a way to ask for a "kind" of training instance from the world in order to maximize meaning. Like maybe I've seen mammals, and now I need to see another kind of animal to maximize my understanding of the meaning animal. A human being—perhaps instinctively, or perhaps by some other force—has the curiosity to go find / pay attention to fish and birds. A kid can tell you he wants to go to the zoo.

A supervised machine learning framework can't tell the researcher what training instances it needs to see in order to improve its meaning. A supervised learning framework can't imagine where it might find that training instance, or describe what it may look like. A supervised learning framework never asks to go to the zoo.

[saulrh]

Yes, in theory an offline supervised learner should never beat an online reinforcement learner. Adding a set of actions A that can be used to bias future examples in a predictable manner is certainly an advantage that will yield better convergence properties in almost all scenarios, simply because it lets you gain more information per observation.