[jerf]

Among the many things I've wished I had time to do, I've wanted to sit down and build a procedural level generator in which the 'tricks' you refer to are first-class citizens, meaning both that they could be incorporated into the levels consistently by the generator and that we could control their introduction into the player's vocabulary consistently.

This approach could be a viable approach to that, but it may need some tuning. It is possible that the problem in this case is less GPT and more the training set; the examples given imply that the levels were characterized as a whole by some very superficial criteria, so it isn't necessarily a surprise that the resulting levels are equally superficial. The system was never trained on "shell jump" (not that that appears in Mario 1 AFAIK, it's just the first Mario term that came to mind), so it never produces them. I would want to look at training on a screen-by-screen basis, with some overlap, rather than levels, and more richly categorizing the input data.

If I were designing a new Indie game, I'd be feeding it some hand-crafted level snippets. However, in terms of getting it out, it would be hard to know whether I can feed the GPT system enough input with enough categorizations to know whether it would just be more cost-effective to design the levels directly. At the moment it is not obvious to me how to convince GPT to understand the concept of level flow, or even something as simply as "this pipe is physically impossible to jump over".

It is also possible there just isn't enough input data to really make this slick. There aren't that many publicly-available Mario levels.