[jacquesm]

That's a good question which deserves an honest answer but this comment box is really too small for that and essay sized comments are frowned upon.

For starters: navigating the web in the beginning consisted of clicking links which caused you to go from one website to another. This all worked well when (a) the web was small and (b) there were (hardly) no trash pages.

Search engines changed that, and once they got 'good enough' the link graph became a mere starting point for crawling the web rather than the way we navigated from site to site. For a little while the link graph was used as a popularity measurement but this too changed (because of the huge number of low value links).

Then we got silos. A 'silo' is a bunch of data locked up under a trade between users and large web properties. The trade is 'you give us your content and a bunch of information about yourself and we'll use that content to attract others and to sell ads'.

Examples of such silos are Google, Yahoo and Facebook.

Finally, if originally (and the internet itself) was strung together by a peer-to-peer approach it turned more and more into a division between producers and consumers, with the producers on the 'server' side and the consumers on the 'client' side.

Mobile devices accessing the net further accelerated this trend, right now the only internet (not web) applications that are still peer-to-peer are torrent applications. For the most part the division on the web is complete and hosting a web server on your very powerful cable modem or DSL line would be grounds for termination of your access.

Servers are hosted centrally and are operated by companies whereas clients are simply terminals that access the content stored on those servers.

I hope that answers your question in enough detail, you could easily write a book about this.

[kbenson]

The internet is, by it's nature, peer to peer and decentralized. Cut a cable, or take out a large networks, the internet will route around it, either quickly (routes converging on a new peer) or slowly (a poorly connected network finding a new upstream to purchase connectivity through). That companies then build on top of this and implement services where they are the middle of both connections does not change this fundamentally, it just adds an optional layer. To assume our connections have upstream bandwidth that is never or rarely used is false. I would argue that we generate more content per-person than ever in history. The seer amount of pictures, videos, webcams, posts and comments is much higher than ever before. Are they hosting it directly from their connections? Usually not, but that's as much a case of being efficient and reaching an audience as it is in companies wanting control over the data. Even then, there are services which are decentralized from that, such as email. It's not efficient to host content yourself. Even the large networks use dedicated CDNs. For the end user, Facebook is a CDN.

That said, I agree there is a clear move towards our data and services being handled by fewer, and larger entities, such as Google, Yahoo, Microsoft, Apple, Amazon. But they aren't a single entity, and I don't consider that centralized. Any one of those providers could implode today, and very little of their services could not be picked up by some competitor easily. I don't consider that centralized.

[jacquesm]

We call them datacenters for a reason. When I received mail in '95 or so the machine receiving it was the workstation I wrote the reply on.

Your peer-to-peer view of the internet died roughly in '98.

[kbenson]

> We call them datacenters for a reason.

And there are many of them, some owned by companies that use them exclusively, some conglomerations of many different providers but owned by yet another party. How is this centralization? I still think you're just arguing that we've compartmentalized certain services to sets of companies, for the most part, but even that isn't centralization, because there are multiple distinct companies using multiple distinct networks and in many cases they are presenting multiple distinct capabilities. Not having something handled at the end point does not mean it's centralized, there's a very large middle ground here, and that's where we are currently at. I'm not sure I see any evidence that we are moving away from that towards actual centralization.

> When I received mail in '95 or so the machine receiving it was the workstation I wrote the reply on.

And many people that used POP3 continued to do so well into the 2000's. It's silly to run a mail server on your workstation. I know, I did it for years myself. You run into all sorts of stupid problems related to your workstation not being always on, badly configured backup MX servers, and other issues. We don't do it anymore not because we were forced out of it (you can still do it now), but because there are solutions that are better for most use cases, and we opt for those.

We don't all wash our own cars, or do our own plumbing, or even clean our own houses. Some people do, some people pay others to do that work. The fact they pay others doesn't mean we've moved towards centralizing those services. There isn't some national bureau of plumbing that is our only recourse when the toilet is clogged and we don't want to fix it ourselves.

[jacquesm]

Ok. So you say we're not trending towards a more centralized internet because you discard all proof that that is exactly what is happening. That's fine with me but it really doesn't help to move the discussion forward.

The reasons why we are moving to a more centralized internet are what is interesting, such as - you rightly identified those - that stuff isn't always powered up and that keeping a mailserver up and running is work and so on.

But none of that changes that centralization is happening.

Multiple distinct companies != peer-to-peer internet. That's what a decentralized network infrastructure used to mean, where the 'peers' were equals.

Nowadays it means clients in one camp and servers in another, and large scale consolidation of those servers in the datawarehouses of a relatively low number of companies serving up the bulk of the data. If that trend continues it's not a bad or a good thing per-se but it would be good to stop and think about how desirable that is.

So from that point of view a lot of centralization has already happened.

Everybody running their own mailserver: could be a good thing, presuming they can be made easy to set up and easy to maintain (I don't see any technical reason why not). Ditto webhosting, why should facebook host all your content (or google, or Yahoo).

In the end, convenience won over 'peer-to-peer', there are many reasons besides convenience (firewalls, for one) but the results are here and we'll have to live with it (except for a couple of die-hard hold-outs).

[kbenson]

What I've tried to make clear, and either failed in or you disagree with this as well, is that I don't think saying we are "centralizing" or moving towards a "centralized" internet is correct, largely because that implies we are approaching, or event still moving towards, the end-point of that spectrum, which is centralization, and that implies a single authority.

I think it is correct to say we are, or at least were, decentralizing, to a degree. I think it's correct to say that we are not fully decentralized, which we were close to initially, but I don't think it's entirely constructive to say we are moving in a direction that leads to a centralized internet, and what that implies (a single authority, even if for a single service). I think we are moving towards, or have arrived at, what we see in many markets. Large dominant players that the majority use, but with a large market of smaller players that provide for the niche needs. Take the automotive industry, for example.

I think we are largely arguing over semantics, which is something I don't want to do, but at the same time it's hard to be sure I'm not just reducing your arguments to the point there's no difference and ignoring important points at the same time.

> But none of that changes that centralization is happening.

I think it's cyclical, and there will be periods where we move along the spectrum back and forth, but I doubt we'll get as close to the decentralized end as we started at, but for many reasons. I don't think we'll get all that close to the decentralized end either though.

> Multiple distinct companies != peer-to-peer internet.

My argument has not been "we are decentralized", it's been "we are not centralized". To that effect, peer-to-peer is irrelevant to my argument, and I've tried to make that clear.

> Everybody running their own mailserver: could be a good thing, presuming they can be made easy to set up and easy to maintain (I don't see any technical reason why not). Ditto webhosting, why should facebook host all your content (or google, or Yahoo).

Because it's very, very inefficient. There are upsides to centralization (e.g. discoverability), just as there are downsides (e.g. homogeneity). I think the sweet spot that maximizes the upsides and minimizes the downsides is somewhere between decentralization and centralization.

[sdenton4]

I think the accurate statement of your opinion is not "the web is centralized" but rather, "Zipf's law sucks."

In decentralized networks there end up being accumulation points, and Zipf's law (which shows up in piles of different contexts, originally noticed in rank of words used in languages) gives a pretty good idea of how that accumulation plays out in basically an L-shaped curve. Point being that it might have a lot more to do with the structure of human networks and attention than with choice of wire protocols...