[chillacy]

> What happens if I have four plates and put one cookie on each?

>> I have four plates and put one cookie on each. The total number of cookies is [24, 5 as a topping and 2 as the filling]

After playing around with AI dungeon for a bit I noticed that the types of mistakes I saw were very reminiscent of common logical errors in human dreams.

For instance in dreams, clocks and signs are inconsistent from one glance to another, location can change suddenly, people come and go abruptly, sometimes I do things for reasons that don't really make sense when I wake up... etc. Things just follow some sort of "dream logic".

[lmcinnes]

I suspect that this is because GPT-2 doesn't have any overarching narrative that it is piecing together. Ultimately it is like a super-powerful Markov based text generator -- predicting what comes next from what has come before. It has longer "memory" than a Markov model, and a lot more complexity, but where a person often formulates a plan for the next few sentences and the direction they should go, GPT-2 doesn't really work that way. And hence it sounds like dream logic because in dreams your brain is just throwing together "what comes next" without an overall plan. Of course your brain is also back-patching and retconning all sorts of stuff in dreams too, but that's a different matter.

[andai]

I wonder if teaching GPT to retcon too would have a meaningful impact on output quality. Right now it does next word prediction one at a time, but what if we ran it again, looking forward rather than back?

Beyond that I am wondering if some sort of logic based AI / goal based AI could be integrated to make it more consistent (or does that still require too much manual fiddling to be useful on large scales?)