[not2b]

But ChatGPT is not a person, it is a text generator. By asking it to generate a new puzzle, you are prompting it to find text in its training data showing someone describing a new puzzle, and it is going to speak in their voice. It's going to emit sentences that were influenced by what the puzzle developer originally wrote, and that person correctly said that it was new.

[Technotroll]

I'm not entirely sure about this. ChatGPT would have to make a model for how such a game was made, and then infer its rules. From that perspective, it would be brand new, although very similar games would perhaps exist out there. And at that point it's also starting to look a lot more like human creativity, although I guess not entirely. As such the statistical or probabilistic approach, or the Chinese room approach, is getting less and less valid for the AI, because it's not doing simple probabilistic look-ups from some table. Instead it's actually developing something "new", or at least with respect to the the perspective of the AI and the data or source material available to it.

[travisjungroth]

I agree with everything you’ve written here, so I’m not sure what the “But” that’s starts your comment is contrasting.

I was answering the question “But how is this different from a person?”. Being asked for something new and finding something that already exists with the word “new” in front of it isn’t normal human behavior. That’s how it’s different from a person.

Zooming out a bit, I think there’s some confusion in this whole chain. There’s a common topic about ChatGPT you could call Question of Creativity. If you ask for a new poem, it just smashes together its patterns around poems. You can debate if this is creativity, and if not, how are humans different. A few comments up, someone brought in a different idea you could call New Matching. If you ask for a new poem it will just grab you a poem that had the words “new poem” in front of it. New Matching is a different idea than Question of Creativity. The person I replied to seemed to be mistaking one idea for the other.

[int_19h]

You're not prompting it to "find text". Comparing the size of the model to the size of the training data is sufficient to conclusively establish that it's an impossibility.

We train it to predict the next word based on the training data, that is true. But we still have no idea what kind of internal structures said training actually produces inside of neural net. It sure as hell isn't just a "stochastic parrot", though, which is rather obvious if you ever tried giving it a complicated multi-step task and solve it while "thinking out loud".

[urbandw311er]

This. people who can ground themselves in what ChatGPT is (an auto completion text predictor) are able to best understand the origins of its output.