[andreyk]

What are you are describing is pretty much reinforcement learning (or learning with access to a query-able knowledge engine, or active learning, or all of these combined). There is work on a bunch of variations of this, but it's true that it's early days for combining it with generative systems.

[namose]

Yeah, I think the query-able knowledge engine is the key here, although I think it’s maybe more like “we haven’t figured out how to generalize conceptual learning”. The computer not only has to be able to query images on the internet, but also know how/what to query, which includes a bunch of actions computers are currently incapable of. In some cases, we might complete a drawing task by traveling to a new place, or taking an action (throw an egg at concrete) not querying the internet.