[inciampati]

You have an idea of what's sensical and not. It would seem that long range correlations in the text break down. This is true of all of the popular models, even the trillion parameter ones. They just break down at longer distances.

[radarsat1]

I suspect this is at least partly because of they way they are used (and maybe trained?)

Always, "given a prompt, keep talking." No instructions to go anywhere, so it's no surprise that they do not.

I think, "start with this idea, end with this one" should give much more interesting results. Telling it to start with a premise and come up with the filler needed to draw some conclusion. It would give it more of a target for making long-distance connections.

Otherwise you just get this open-loop blabbering, which I agree seems really useless. With a more "directed" model I can see this having actual applications, like with story writing or interactive video games. But as it stands this seems totally uninteresting from an applications point of view.