[loser777]

I love bringing up Pokemon as a humbling example of how AI progress is counterintuitive. Naively we might see games like Go, Chess, StarCraft, and Dota as more "complex," yet current approaches still tend to fail here, a game we would expect an early-grade school child to have no problem completing because of how incredibly sparse the reward function is. I hope that one day we'll get a satisfying tabula rasa solution to narrative-based/world model games like Pokemon rather than something like "well it turns out a GPT ingested a gameplay walkthrough during training and can regurgitate that as gameplay inputs womp womp."

[comex]

To be fair, even those early-grade-school children come in with the ability to read the text, while game-playing AI usually does not. If a human tried to play Pokémon Red without being able to read anything (even the numbers!), they would probably succeed eventually, but it would be quite difficult and frustrating. And even without reading, the human would still be making inferences based on how the graphics and sound resemble real-world objects. So for an AI to play the game “properly”, it really shouldn’t be a complete tabula rasa; it should have some of that knowledge too.

[Aachen]

> even those early-grade-school children come in with the ability to read the text, while game-playing AI usually does not.

You sure? :D No idea what a grade school is, but I couldn't read a word of English at the age that I played Pokémon on the gameboy!

I did need occasional hints (every couple weeks) from my older cousin to get unstuck, but afaik still loved the game, I guess just because it's about Pokémon. I think cut (meaning both pussy and fuck in Dutch, fun fact) was one of the things I got stuck at, perhaps needing to teach it to a Pokémon, but probably on how to use it. It's a number of different things to connect and so you can't button mash your way through, iirc (it has been a few decades)

[Auracle]

Quick anecdote: a nephew of mine, who was in third grade at the time, was playing my brother-in-law’s old Gameboy and some version of Pokémon.

He wasn’t reading the text. He would then get stuck, not knowing what to do, and start over. He did this several times and complained that the game was broken.

Some time later I gave him Pokémon Sword for his Switch, as I thought it’d be good for him. Some time after that he was diagnosed with ADD (to the surprise of no one). Some time after that he proudly told me he beat it.

He also said he didn’t like it that much. Hah. I’ll still count it as a win.

[gwern]

Go-Explore could probably solve this, but yeah, I wouldn't describe that as a satisfying solution yet. DRL has been in hibernation because generative AI has sucked all the oxygen out of the room, but I remain optimistic about directions like Gato working.

[sapiogram]

> Naively we might see games like Go, Chess, StarCraft, and Dota as more "complex"

The strong AIs made for those games had access to some crucial human "help" though.

* AlphaStar (For StarCraft) was trained on a massive database of human games, while this project started Pokemon Red with a blank slate.

* AlphaZero and AlphaZero Chess did start blank slate, but had a hand-coded and domain-specific search algorithm (MCTS) to explore various actions (800 iterations IIRC) before actually making that action. There's no obvious way to do something similar for Pokemon Red.