[lsc]

>In reality, having an IP address does not put you on an equal footing --- in a service model sense --- with other servers or companies that have paid massive amounts of money for peering. BigCo IP addresses are already "super-addresses", because they're BGP-advertiseable, and yours aren't.

Ok, I often find I need to check myself before calling someone out on hacker news... several times I've found myself arguing with someone who was way more qualified than I was on the subject at hand.

but this is the exact opposite of what I understand "peering" to mean. From what I understand, settlement free peering is just that... it's free. Each party pays for half the cost of maintaining the line between them and packets bound from the customers of one peer to the other and vis-a-vis can traverse that link for free.

Generally speaking, when you pay for transit, it's called transit rather than peering.

(Now, my understanding is that there are cases where you would pay for peering... in this case, say you want to shave miliseconds off your ping time to some stock exchange... you can essentially buy transit that is limited to just the customers of your peer. In this case, there is "settlement" based on the number of packets going one way or the other. But, my understanding is that this isn't how it's usually done. Normally you look people up on peeringDB, and if you are exchanging enough traffic for it to make sense and you are on the same exchange, you set up a settlement-free peering agreement. Of course, all this shit is covered by NDA, so all we really have is hearsay)

Also, uh, all IPs are BGP advertisable. If you are small, your ISP does the BGP advertising (and the peering) I mean, if I buy my connectivity from above.net, they peer with everyone, right? so that's pretty close to me peering with everyone. Now, if you buy connectivity from a poorly connected ISP, sure, your network is going to be slightly slower... but it's still certainly BGP-advertiseable - it's just that you've outsourced the management of that BGP advertisement to your ISP.

Right now I'm working on moving the BGP router into my control, so I'm getting first hand experience with things I've watched people do in the past. And really, unless your primary business is infrastructure (and, well, mine is) it often doesn't make sense for you to run your own BGP router. I mean, it's one of those easy to screw up things. Most places I've worked that did their own BGP had more outages due to the new guy jacking with the router than due to upstream outages (which controlling your own BGP router, assuming you have multiple transit providers, can protect you from.)

[tptacek]

Say you want to run an app out of your house. You can trivially afford both DSL and cable. Can you multihome your app using IPv4?

Your IP addresses aren't (in all likelihood) portable. If you have less than a certain number of addresses, they're actually not advertiseable, because ISPs will filter smaller announcements. Even if you have a portable allocation, you may find it difficult (ie, expensive) to get your ISP to advertise it.

It is obviously possible to overcome all these problems with skillful application of money, but that's my point: the IP address itself isn't giving you this power, but rather the juice you pay to your ISP to make that happen.

(It's been over 10 years since I had to configure default-free BGP4 anywhere, though I've spent a lot of time working with BGP4 since then. Feel free to call me out on any of this.)

Now consider your cable connection, your DSL connection, and BitTorrent. BitTorrent can trivially scale across multiple Internet connections. You don't (heh) have to ask your ISP for permission to multihome it. That's because BitTorrent lifts the task of endpoint rendezvous out of IP and up to the app layer.

The future belongs to things like BitTorrent, where the average user never has to care whether packets are being carried by IPv4 native, IPv4 NAT, IPv6, or carrier pidgeons.

[lsc]

>Your IP addresses aren't (in all likelihood) portable. If you have less than a certain number of addresses, they're actually not advertiseable, because ISPs will filter smaller announcements. Even if you have a portable allocation, you may find it difficult (ie, expensive) to get your ISP to advertise it.

Yes, my home IP addresses are not portable... but IP portability doesn't matter until you have a large number of IP addresses. Ok, so I have to renumber the 8 servers I have in the garage, big deal. Now, if I have to renumber the 1000+ virtuals I've got in the data center, that is a big deal. But my understanding is that everyone filters what's smaller than a /24, and some people filter below a /22... but at a bit over 1000, you are pushing a /22, so by the time that renumbering becomes a really impossible task, nobody filters you any longer.

Now, if you are multihoming for reliability rather than ease of switching ISPs, yeah, you have a good point. DSL has been neutered in that regard, which is really fucking irritating... but I understand why they did it. You know as well as I do that every route eats up a few bytes of ram in everyone else's BGP router, and TCAM is fucking expensive, so having a grand a month barrier before you can start adding your data to every bgp router in the world seems reasonable to me.

On the other hand, the barrier to getting some data center space and a multihomed /24 is pretty trivial by bay area sysadmin salary standards. Over the next week, that's one of my projects. One of my customer has an old /24 of swamp space he wants me to announce. It sounds like a good deal all around just 'cause my BGP foo was never good and what experience I have is old, so I get to practice on something I can break, and he gets to play in his swamp. It's going to cost the guy nothing 'cause I want to play around... It'd probably cost him something around a grand a month if he was going with someone more professional than I am and/or if he didn't already have the swamp space.

But my point was that if you pay for an ISP with good connectivity, you get all the benefits of their peering efforts. You are outsourcing your BGP management to your ISP, which for most people is going to result in better service than doing it themselves.

I think you can get 90% of the benefits of running your own BGP with a lot less hassle out of a $200/month co-lo plan. (just to be clear, that won't get you BGP you control or portable addresses, but if you've only got a few addresses, and the isp you chose is competent, well, that doesn't matter all that much for at least 90% of use cases.)

Edit: I know little of the bittorrent protocol... but my understanding, and this may be incorrect, was that bittorrent from one computer behind a nat (without upnp or the like) to another computer behind another nat (again without upnp or a port forward or the like) did not work well or at all... if you were behind a nat without a port forward (or upnp) you could only talk to peers that were on a public IP or had a port forward.

[tptacek]

This thread is officially unwieldy. I think you know what I'm saying now (even if you don't agree with it). My contact info is pretty easy to find.