[unnouinceput]

Yes it is. Same with flappy birds dude from Tencent. Try to do a simple game that is less found on internet, some obscure board game and you'll see how ChatGPT fails spectacularly. For those who knows Witcher 3 game, try asking ChatGPT to implement Gwent board game found within W3 world, or try to implement the board game Orlog that is found within the world of Assassin's Creed Valhalla game. See those implementations how they go - no need to do a GTA complex style one.

[TeMPOraL]

I don't see why you'd fail. With a more obscure game, you need to elaborate on details some more. E.g. describe the specific rule instead of saying "like in Angry Birds".

[jhoelzel]

Well the fact is that it does not really learn. So yes you go ahead and explain in detail what will be forgotten once the context exceeds 8k tokens

[TeMPOraL]

Sure, but no one says you have to get it to write you the whole game in one session. Divide and conquer! As long as you keep the work items small enough to solve them within 8k context limit, GPT-4 will be able to help you with all of it!

Creating a collection of system prompts to select from, as well as plain old copy-paste, are two tools you can use to provide the same context to multiple sessions, with little to zero effort.

[ben_w]

I'm trying something like this with 3.5; the problem is that it keeps changing its mind about what the API "should" look like for the other parts it can't remember in context. As I'm using it as a collaborator, this is actually fine, but it wouldn't work if I was trying to 1-shot it or otherwise use pure LLM output.

[TeMPOraL]

I wouldn't expect this to work well with GPT-3.5. And, with GPT-4 available, I wouldn't bother trying; there's too big a capability difference between the two. However, it's nice to know 3.5 is still somewhat useful here.

[jhoelzel]

yes and no. it will forget about your properties and structs just the same way it will forget about keys design decisions.

[TeMPOraL]

Within a session, not unless you stay under context window limit. Between sessions, it's up to the user to carry over relevant context.

[jhoelzel]

It would be very interesting to see a post about this then please. For me it has definitely been an issue that anything meaningful will take multiple files and henceforth cap my input for it.