[G_z9]

This allegory has haunted me for ten years. I always see it posted here and invariably the poster doesn’t elaborate or explain at all. Im pretty sure the people who post this just do it to look smart. I can’t find an explanation anywhere on the web. I asked chatgtp what it means and it said this:

This allegory, often referred to as an "AI koan," is a story that conveys a deeper meaning about the nature of artificial intelligence and the process of learning.

In the story, Sussman is a novice who is attempting to train a neural net to play Tic-Tac-Toe. When Minsky, a renowned AI researcher, asks why the net is wired randomly, Sussman responds that he does not want the net to have any preconceptions of how to play. Minsky then closes his eyes, explaining that he is doing so in order to empty the room.

The meaning of this story is open to interpretation, but one possible interpretation is that it is highlighting the importance of approaching problems with an open mind, free of preconceived notions and biases. By wiring the neural net randomly, Sussman is allowing it to learn through trial and error, without being constrained by prior assumptions about the game. Similarly, by closing his eyes, Minsky is symbolically "emptying the room" of preconceptions and biases, allowing himself to approach the problem with fresh eyes and an open mind.

Overall, the story encourages us to approach complex problems with a beginner's mind, free of preconceptions and biases, in order to allow for creative solutions and new insights to emerge.

Back to human commentary: I’m not sure that makes sense. will someone please explain this stupid allegory and let me finally rest?

[bigDinosaur]

I can't speak to the technical details but my basic interpretation was the opposite of what ChatGPT just said - no real-world agent ever approaches a problem with zero preconceptions. That's simply not how learning works for humans. Presumably AI models also have 'preconceptions' at least in how they are designed.

[tempodox]

The way the net is wired is the preconception. Just as closing your eyes doesn't actually empty the room, randomizing the preconception doesn't make it go away.

[ayewo]

The room doesn't become empty by the mere act of closing one's eyes.

Closing one's eyes temporarily pauses sensory input from the room to one's brain (through the eyes). To fully pause sensory input to one's brain, one will also need to block the ears (from picking up sounds) and perhaps stand aloof from others in the room (to avoid touching any surfaces or being touched by others).

Even if it were possible to block all senses: visual (via the eyes), auditory (via the ears), olfactory (via the nose) and haptic (via the hands) input, this doesn't make the room become empty.

Put differently, reality (what is there) and our perception of reality (what we perceive) are two distinct concepts that are easy to conflate.

So, for anyone to claim they were successful at removing preconceptions or bias from a neural net, in a way that can be independently verified by others, they will first have to enumerate all forms of bias known to man, then show that all of those biases were avoided in the programming of the neural net.

At least, that's how I understand the koan.

[kgwgk]

I take it to mean that a random wiring still represents some (random) preconception even though we don’t know it - just like the world still exists when you close your eyes even if you don’t see it.

I could be getting it wrong, maybe ChatGPT is more intelligent than me…

[G_z9]

Maybe Marvin is saying that there might be advantageous patterns in the random values, and that the researcher is only selecting for what he can see but not necessarily what exists, since many chess algorithms might just look like random noise to a human. So the point of the allegory is that intelligence is much broader algorithmically than what human intuition is able to grasp.

[krackers]

I think you have to loosely interpret "randomly wired" as "fully connected", i.e. no predefined structure (because an actually randomly wired net probably wouldn't learn well). The thing is as we've seen with convnets and now transformers, the structure of the network actually matters quite a bit. Even though a fully connected network could theoretically learn the right weights so that it emulates a convnet, in practice this is too hard to do. See the discussion at https://news.ycombinator.com/item?id=34748190