[tptacek]

You realize that just like 64 bit addresses presume we'll never have a 18446744073709551616th host, 128 bit addresses presume we'll never have another network layer, right?

Have you considered whether we're putting entirely too much emphasis on whether or not there is an end-to-end "Internet Protocol" Internet with a single coherent address space? Most people don't use Internet addresses. In fact: most people already use an entirely application-intermediated address format: it's called "the first 4 results on a Google SERP".

[lsc]

right. but the old internet... the way it used to be was that there was no real difference between a "client" and a "server" - an IP address was an IP address. Yeah, for most people, the Internet is the web, and that's enough. And nat works great for that.

The problem with enforcing this idea (that you have "clients" with private addresses behind a NAT and "servers" with public addresses) is that this won't be able to change. Doing peer to peer gaming, filesharing, video chat, etc... through multi-later (or carrier) nat is very difficult. It works through our home NATs right now, well, sortof, because there is one public IP for 2 or 3 computers. Upnp and other tricks can usually handle getting through a single layer nat with only a few private IPs behind it. If your ISP owns the nat and has hundreds of people behind the same v4 public IP, or in a double layer nat, where the ISP gives you a private IP behind a nat (that obviously doesn't support Upnp) suddenly this is going to work a lot less well.

Now, you can solve the problem by just making everything client-server. Want to video chat or game? each person needs to connect to a server with a public IP and talk through that server. It's doable, but it means that the Internet the next generation grows up with will be a different sort of network than the Internet I grew up with. This network will be one where you are either a "server" or a "consumer"

Some people say we've been moving in this direction for a long time, but this doesn't make it any less sad.

(also note... it's not just nerds that get screwed... if anything, gamers will get screwed more than nerds. Nerds can get VPSs that work just fine for most anything a nerd would want to do. VPSs, generally speaking, make shitty game servers, though.)

[tptacek]

This is exactly the opposite of reality.

In reality, having an IP address does not put you on an equal footing --- in a service model sense --- with other servers or companies that have paid massive amounts of money for peering. BigCo IP addresses are already "super-addresses", because they're BGP-advertiseable, and yours aren't.

So long as "full membership in the Internet" means "publicly routable IP address", you're going to get what your ISP is willing to give you and nothing more. This is true even in an IPv6 world! I'm not comfortable with this and you shouldn't be either. IP addresses are what network operators are giving greybeards to geek out over while they continue gobbling up the Internet.

What we need to do is accept an IPv4/NAT IP layer, define a minimum acceptable service model for ISPs to offer over it ("access to the web" being a good starting point), and then build application-layer overlay networks that provide the real services applications want, like broadcasting, peer-to-peer, location, presence, automatic configuration, and multihoming.

This isn't my crazy pie-in-the-sky idea (though the first startup I personally cofounded got this idea funded for several million dollars during the bubble). Is also the MIT PDOS RON idea, which Paul Graham's friend Robert Morris helped oversee.

It is also, for what it's worth, the logical conclusion of Saltzer and Reed's "End to End Argument In Systems Design". When you meet a challenge with a lower-level protocol, the answer tends to be to dumb it down to a point where you can build multiple variants of "something smarter" on top of it. We're at a point now where IP is simultaneously getting less relevant (organically, as more intelligence moves into HTTP-driven protocols) and more important (as we run out of addresses). The answer is not investing more effort in IP.

From a pragmatic perspective, the nice thing about this strategy is that it requires nothing from normal people. They'll use whatever IP their ISP gives them (NAT'd or otherwise), and it won't matter; it'll work just fine for the web today, and it'll work just fine for the TCP/SCTP/whatever-driven overlay networks we come up with tomorrow, where all the real action will be anyways. It's also nice to sit back and not worry about the IPv4acolypse and concentrate on building stuff instead.

[lsc]

>In reality, having an IP address does not put you on an equal footing --- in a service model sense --- with other servers or companies that have paid massive amounts of money for peering. BigCo IP addresses are already "super-addresses", because they're BGP-advertiseable, and yours aren't.

Ok, I often find I need to check myself before calling someone out on hacker news... several times I've found myself arguing with someone who was way more qualified than I was on the subject at hand.

but this is the exact opposite of what I understand "peering" to mean. From what I understand, settlement free peering is just that... it's free. Each party pays for half the cost of maintaining the line between them and packets bound from the customers of one peer to the other and vis-a-vis can traverse that link for free.

Generally speaking, when you pay for transit, it's called transit rather than peering.

(Now, my understanding is that there are cases where you would pay for peering... in this case, say you want to shave miliseconds off your ping time to some stock exchange... you can essentially buy transit that is limited to just the customers of your peer. In this case, there is "settlement" based on the number of packets going one way or the other. But, my understanding is that this isn't how it's usually done. Normally you look people up on peeringDB, and if you are exchanging enough traffic for it to make sense and you are on the same exchange, you set up a settlement-free peering agreement. Of course, all this shit is covered by NDA, so all we really have is hearsay)

Also, uh, all IPs are BGP advertisable. If you are small, your ISP does the BGP advertising (and the peering) I mean, if I buy my connectivity from above.net, they peer with everyone, right? so that's pretty close to me peering with everyone. Now, if you buy connectivity from a poorly connected ISP, sure, your network is going to be slightly slower... but it's still certainly BGP-advertiseable - it's just that you've outsourced the management of that BGP advertisement to your ISP.

Right now I'm working on moving the BGP router into my control, so I'm getting first hand experience with things I've watched people do in the past. And really, unless your primary business is infrastructure (and, well, mine is) it often doesn't make sense for you to run your own BGP router. I mean, it's one of those easy to screw up things. Most places I've worked that did their own BGP had more outages due to the new guy jacking with the router than due to upstream outages (which controlling your own BGP router, assuming you have multiple transit providers, can protect you from.)

[tptacek]

Say you want to run an app out of your house. You can trivially afford both DSL and cable. Can you multihome your app using IPv4?

Your IP addresses aren't (in all likelihood) portable. If you have less than a certain number of addresses, they're actually not advertiseable, because ISPs will filter smaller announcements. Even if you have a portable allocation, you may find it difficult (ie, expensive) to get your ISP to advertise it.

It is obviously possible to overcome all these problems with skillful application of money, but that's my point: the IP address itself isn't giving you this power, but rather the juice you pay to your ISP to make that happen.

(It's been over 10 years since I had to configure default-free BGP4 anywhere, though I've spent a lot of time working with BGP4 since then. Feel free to call me out on any of this.)

Now consider your cable connection, your DSL connection, and BitTorrent. BitTorrent can trivially scale across multiple Internet connections. You don't (heh) have to ask your ISP for permission to multihome it. That's because BitTorrent lifts the task of endpoint rendezvous out of IP and up to the app layer.

The future belongs to things like BitTorrent, where the average user never has to care whether packets are being carried by IPv4 native, IPv4 NAT, IPv6, or carrier pidgeons.

[lsc]

>Your IP addresses aren't (in all likelihood) portable. If you have less than a certain number of addresses, they're actually not advertiseable, because ISPs will filter smaller announcements. Even if you have a portable allocation, you may find it difficult (ie, expensive) to get your ISP to advertise it.

Yes, my home IP addresses are not portable... but IP portability doesn't matter until you have a large number of IP addresses. Ok, so I have to renumber the 8 servers I have in the garage, big deal. Now, if I have to renumber the 1000+ virtuals I've got in the data center, that is a big deal. But my understanding is that everyone filters what's smaller than a /24, and some people filter below a /22... but at a bit over 1000, you are pushing a /22, so by the time that renumbering becomes a really impossible task, nobody filters you any longer.

Now, if you are multihoming for reliability rather than ease of switching ISPs, yeah, you have a good point. DSL has been neutered in that regard, which is really fucking irritating... but I understand why they did it. You know as well as I do that every route eats up a few bytes of ram in everyone else's BGP router, and TCAM is fucking expensive, so having a grand a month barrier before you can start adding your data to every bgp router in the world seems reasonable to me.

On the other hand, the barrier to getting some data center space and a multihomed /24 is pretty trivial by bay area sysadmin salary standards. Over the next week, that's one of my projects. One of my customer has an old /24 of swamp space he wants me to announce. It sounds like a good deal all around just 'cause my BGP foo was never good and what experience I have is old, so I get to practice on something I can break, and he gets to play in his swamp. It's going to cost the guy nothing 'cause I want to play around... It'd probably cost him something around a grand a month if he was going with someone more professional than I am and/or if he didn't already have the swamp space.

But my point was that if you pay for an ISP with good connectivity, you get all the benefits of their peering efforts. You are outsourcing your BGP management to your ISP, which for most people is going to result in better service than doing it themselves.

I think you can get 90% of the benefits of running your own BGP with a lot less hassle out of a $200/month co-lo plan. (just to be clear, that won't get you BGP you control or portable addresses, but if you've only got a few addresses, and the isp you chose is competent, well, that doesn't matter all that much for at least 90% of use cases.)

Edit: I know little of the bittorrent protocol... but my understanding, and this may be incorrect, was that bittorrent from one computer behind a nat (without upnp or the like) to another computer behind another nat (again without upnp or a port forward or the like) did not work well or at all... if you were behind a nat without a port forward (or upnp) you could only talk to peers that were on a public IP or had a port forward.

[tptacek]

This thread is officially unwieldy. I think you know what I'm saying now (even if you don't agree with it). My contact info is pretty easy to find.