[Misdicorl]

Time for my pet theory!

Search is orthogonal to exploration. At first Google/yahoo/etc we're so bad that improvements in their systems made everything better.

But at some point they started trading one against the other because the easy wins were gone.

Because Google is the market leader (at least by the time this becomes relevant) everyone tried to compete with them and follow their path. Google optimized search really strongly over exploration. So much so that most users only interact with the very first result these days.

And thus exploring the internet is dead because there's not a good way to do it. This had knock on effects where nobody designs to be explored like they used to (or are explicitly hostile to it), so it's probably actually quite hard to build an internet explore engine at this point

[thex10]

This is basically my theory too!

Put simply, the dominating means of navigation and discovery on the internet switched from directories (transparent, easy to explore) to search engines (black box, returns result tailored to your input). And each website you ended up at might have had its own directory inviting you to explore further! Though maybe that bit hasn't changed too much.