Hacker News new | ask | show | jobs
by bkyiuuMbF 1546 days ago
With acknowledgement to the palette hack discussed in the other comments, I still think there's a ton of value in this.

So often, people observe patterns in nature that appear to be so unlikely as to be by design. I have family members that are superstitious: if a light flickers at the same time that they mention a recently deceased loved one, it must be "a sign". Similarly, they will point to some overwhelmingly unlikely occurrence in the news, and ask, "how do you explain that?" The answer is usually random chance (if not deceit). And this exercise is a good illustration of that.

"Waldo, in the digits of pi?!? What are the odds??" - To the outsider who is not scientifically minded, this just looks so coincidental as to be magic. But almost any pattern can be found in randomness. It's just that the size of the necessary search space explodes as the query becomes larger / more specific.

The average HN reader knows all this, but the Waldo story is still a cool way to describe it to other people.

6 comments

This post reminds me of the once common meme[1] that pi contains all things you will see in your life, all books you will ever read and all songs you will ever hear, the first thing you saw when you were born, and the last thing you will see before you go away for one final time.

Off course, pi isn't proven to be normal (containing all possible sequences of decimal digits, if I remember that correctly) yet so this thought experiment needs an encoding aware of that to work, and any random bit source that unrepeatingly explores the combinatorial space of a symbolic alphabet would work, pi is not special here at all. This is just the library of babel + encoding arbitary data structures into numbers.

As you say, finding patterns is not hard, it's unsurprising that random strings contain those things, maximum-entropy information sources maximize the expected information as per Shannon. The whole point, though, is finding useful patterns: things that predict other things you care about, which you didn't know beforehand (or just knew in vague outlines), and which are cheaper to find and explore than simply directly observing/simulating the things they predict. None of this holds for the 'patterns' you will find in a typical library of babel. Evolution and evolutionary algorithms can be seen as a tool to cull the impossibly large search space of a library of babel. Also mental heuristics like Occam's razor can be seen as tree-pruning heuristics this way.

An even more beautiful idea than "An infinite random string contains all possible data structures" is "An infinite random string contains all possible programs and all possible computations (according to all possible semantics of those programs)", explored by the legendary Greg Egan in Permutation City. I can't do justice to Egan in this already too-long-of-a-comment, but I promise you will absolutely be mind blown.

[1] https://slate.com/technology/2013/04/pi-meme-on-reddit-and-g...

There are an infinite number of coincidences happening in every moment: a raindrop falling, a light turning off, a wind blowing on mars, a star rotating, and so on.

If one cared to find "something apparently significant" in any moment, one would therefore find an infinity of them.

By comparison, events which are directly causally connected are diminishingly few, and mostly indistinguishable from those coincidences. Hence, most things are actually unknowable, and what few beyond the ordinary, require extremely expensive and technologically advanced science to uncover.

You know, the most amazing thing happened to me tonight... I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!
> What are the odds?

Assuming pi is normal [1], the probability of any bit string occurring in pi is 1.

[1] https://en.wikipedia.org/wiki/Normal_number

But the probability of you actually finding where that string occurs before the sun goes nova is much, much smaller.
> But almost any pattern can be found in randomness.

Not quite. Although this is a popular misconception. It depends on the cardinality of randomness and the distribution that you're talking about.

As a casual example, an infinitely long randomly distributed sequence of set { A, B, C, D } will never contain the string "bkyiuuMbF."

You can take this example and apply it to other spaces. For instance, we may live in a universe where there really is no other set of possible interactions where life exists somewhere else in this supercluster. Maybe in a another one, though?

There are nearly 8 billion people on the planet at the time of writing, I think. There's a lot of random people in there, but only one you!

Part of this though is also that we are optimized to see faces.