[samwillis]

Ok, so maybe not the issue with ChatGPT, but with peoples understanding of its limitations. It can generate text and code from instructions, but it's limited in its logical analysis of what it's "saying". In this case it was asked:

> And to the best of you knowledge this type of puzzle does not currently exist?

and it responded:

> As far as I am aware, this specific type of puzzle with the given rules and mechanics does not currently exist in the puzzle game genre. However, there may be similar games out there that share some similarities with this puzzle.

That response is not generated (as far as I am aware) by any form of logical analysis or understanding, it's just generated text based on its training and prompting. It was asked to come up with something "new", and will continue to claim that as it was part of its prompts.

So yes, this may not be a failing of ChatGPT, but of users understanding of it. You cannot take what it states as "fact" as anything other than potential BS. But it is an incredible tool for using to generate text and code.

We are still early in its development though, who knows where it will be in 18 months time!

[kqr]

This is a very good comment. ChatGPT uses language so fluidly it's easy to interpret as there being more substance than there is.

Looking at the response the way you suggest, it's clear it's given a boilerplate answer that would seem likely given the context it has found itself in.

[prox]

Exactly, as soon as were to butcher my english, a certain amount of credibility will be incurred, even if the communicators are aware of it. It could probably eloquently explain the workings of a Retro Encabulator fluidly and you’d nod a few times and thinking it’s fine.

If words not right said for listen like now, think you might not be smart as is tho.

[Jensson]

ChatGPT will reverse that, if you sound smart it is likely nonsense generated by an AI.

[sebzim4500]

Is it actually wrong though? Will the rules of 'summer' be in it's training data anywhere? AFAICT they aren't described on the google play page, although you can easily figure them out by the screenshots.

[vsareto]

I feel like you can compensate with more complicated prompts. Or even different prompt categories (like negative prompts, but for programming it might be a list of constraints). Like this interface: https://github.com/AUTOMATIC1111/stable-diffusion-webui but for code