[zbisch]

Ah, very good point. The last thing I read on the Level3/Comcast spat story lead me to believe Comcast was paying Level3 for the peering. I apologize. However, this line lead me to believe the author had a misunderstanding of the costs of traffic:

"...they almost certainly traverse fewer fiber route-miles and physical router ports (Comcast’s two primary costs of delivery)."

That is, unless the author meant Comcast's primary costs within their network, which is not what I read it as. Without providing reference to the cost of the peering, the comparing the _geographic_ distance of traffic seemed irrelevant. Even in this case, the reason that it is more expensive for Comcast to send traffic is not that is coming from Seattle, but because they are losing money on having Level3 pay them.

Edit: I also need to read more on this as I'm curious why Level3 is paying Comcast since Comcast is (as far as I know) not acting as a transit network. But I suppose this is what happens when you violate valley-free routing.

Edit2: Also, just wanted to add, the fact that Comcast controls the peering links is why this is so sketchy. By raising the costs of peering links, they could make it way more expensive for the competitors to send their users traffic, forcing the competitors to raise their prices, allowing Comcast to be the only content provider to have reasonable prices. While the part about the bandwidth to them not counting towards your cap is sketchy, the practice of charging you less, a lower flat rate, or not at all for traffic that stays within the network is reasonable. Where it is sketchy is when that difference is leveraged to make the service cheaper. If the service is more expensive, it's the fact they are charging competitors more that is sketchy. So... in general, providing content and the network is all around sketchy.

[bjornsing]

Just a quick comment on cost (since OP and these comments seem to have it a bit backwards):

The real cost in a DOCSIS network like Comcast's is actually in the last mile. DOCSIS is sort of like a mobile network in a pipe. The base station in this case is called a Cable Modem Termination System (CMTS) and data is transmitted from there over RF with QAM modulation to all the cable modems on the same coaxial segment. This can be hundreds or even thousands of cable modems. For unicast IP the packet is only received by one of them but they all have to listen.

When the radio spectrum inside the coaxial cable becomes congested the segment has to be split in two parts and a new CMTS installed, very similar to how you split a mobile network cell into sectors to improve capacity. Like with a mobile network this is capital intensive.

Not saying this makes Comcast right. But important to understand where they're coming from.

[zbisch]

Residential networks are actually part of my current research, and, although I don't know the full details of every technology, I am familiar with the basics of how DSL and DOCSIS networks work.

Yes, there is a cost (in terms of performance and congestion) over these links, but this not really relevant in the discussion when talking about the cost of where the traffic is originating. Whether the traffic originates in Comcast's data center in Seattle, from a CDN in Level3's network, or over their peering links with Tata, the cost of it traversing the last-mile is irrelevant. In any of these cases, traffic will still be crossing this link. And yes, this is a significant cost in terms of deploying the last-mile network, but as I said, it has nothing to do with the cost of getting of getting the packets to the DOCSIS network.

What we're saying is that if Comcast is carrying traffic from Seattle to a user in their network, they do not have to pay a transit provider to carry this traffic. If it is from Level 3, Level 3 would actually be paying them to carry that traffic to Comcast subscriber. If the traffic were coming from Comcast's peering link with Tata, Comcast would have to pay Tata (their transit provider) for the traffic.

Yes, the cost of the last-mile network is real and important, however, in this discussion it doesn't really relate, since you would be traversing the DOCSIS network in all cases.

[bjornsing]

Well... Isn't there a fundamental assumption in the network neutrality debate that cost per bit is zero or very low? At least that's the feeling I get from reading this discussion. People seem to think that cost comes mostly from IP transit, and implicitly that last-mile infrastructure cost is constant and sunk.

Well, that assumption is wrong for mobile, and to some extent for DOCSIS. Last-mile cost scales with traffic volume (and completely overshadows the cost of IP transit). That's why I think you'll see MNOs and MSOs fighting the hardest against network neutrality.

[wmf]

Level 3 is paying Comcast because the alternative was for Comcast to let the peering links congest, which would hurt Netflix streaming performance. Basically Comcast has a new business model for their Internet service: Everyone pays Comcast and Comcast pays no one.

[zbisch]

Yeah I can't believe this works for them. So I assume Comcast pays Tata, right? I was under the impression at some point that was their primary provider (paid transit). They keep these links so congested [1], that in order to actually provide reasonable service to Comcast users, a content provider needs to pay to peer with them (or pay someone else to) in order to give reasonable performance. I guess if you have enough users and enough of a demand to reach them faster, you can charge others. I take this to mean that Comcast should be paying me to subscribe to their Internet service! (or at least to watch Netflix on their network)

[1] http://www.merit.edu/mail.archives/nanog/msg15911.html

Edit: Although it is not my exact area of research, a colleague of mine has done some interesting work that discusses the costs of P2P traffic for some of these ISPs that is somewhat related: http://torrentfreak.com/large-isps-profit-fom-bittorrent-tra...