Trouble is that NAT solved a huge chunk of the problem. The current setup is not quite painful enough for people to want to fix it. The good is the enemy of the great, as it were.
NAT was (and is) destructive and painful that some of us gave up writing network software in the late-90s/early-00s. I personally abandoned several network-focused projects in the early 2000s.
The current status quo only seems "not quite painful enough" if you accept that most people cannot use true network software, limited to client-server architecture where party lines[1] communicate with each other only with the permission of central privileged imprimatur[2].
"if you accept..." I mean, isn't that a very reasonable assumption? Do you really disagree with the notion that more than half of global population don't care at all about the ability to "use true network software", wouldn't use it if they could, and, as long as they're not restricted too much, knowingly avoid solutions with more freedom and actually prefer centralized solutions as long as they're even just a bit more convenient? Heck, if we don't listen at what people are claiming and look at their actions, then even in the techie crowd the majority aren't ready to sacrifice minor conveniences to choose decentralized models over a client-server run and entirely controlled by someone else.
Interrent would be very different (better) today if not for NAT.
NAT restricts what is possible to do over network for example we only use TCP and UDP protocols, because those protocols are supported by most devices. Similarly we have very minimal number of peer to peer applications. P2P currently is mostly popular with piracy, but it could be beneficial for other uses.
NAT + asymmetric speeds (which started because of DSL, but ISPs decided to keep things that way even though it is no longer necessary) are responsible that's why we haven't a lot of services that are centralized. IPv6 has chance to fix this and I am so glad NAT wasn't included in its design.
ipv6 solve the problem of one ip per device, with ipv6 you can have both local and public IP's on the same interface! Then your app's and services can choose whether they like to listen on local or public IP's.
Without NAT you can do so much more stuff, like peer-to-peer (p2p) networking. Yes, you can do p2p with ipv4 behind NAT but it's super complicated and brittle.
Also bypassing the NAT is complicated, you have to fiddle with the router settings, and often you have to call your ISP to give you a public IP. This makes it hard or impossible to sell "Internet of things" (IoT) devices to regular people as you can't just plug them in.
Networks today are very good with high bandwidth and low latency, which enables some interesting use cases, for example virtual reality (VR) where you just have a thin client plugged in to the network and then have all the compute power located in a data-center a few miles away, with sub ms latency.
Another usecase is apps with service like functionality, like decentralized Facebook, and chat messengers.
Do you have redundant power supply at home, redundant internet connection? Keeping your own server up and running at home is unreliable and annoying. Having animals, kids, makes it even more difficult. If I would have to rely on it beeing up while I am abroad, I would rather pay for VPS.
Hiding insecurity is perfectly valid. It is making attack surface smaller. I do not get pings of death, constant scanning, login attempts all the time on my local machine which is always behind NAT. Every server that has public IP gets scanned or tried out with vulnerabilities. I can connect totally new PC to router with NAT and not be owned in matters of minutes by some botnet. My router might be exposed but it is something I know. All machines behind router are perfectly fine for remote vulnerabilities.
> Do you have redundant power supply at home, redundant internet connection?
Depends what you need. My last power outage was over a year ago, and Internet issues will generally resolve themselves in a relatively short period of time. That's reliable enough for a lot of use cases.
> Do you have redundant power supply at home, redundant internet connection? Keeping your own server up and running at home is unreliable and annoying.
That's all besides the point. When you want to share a file with someone while you are both working on it, say, there is no need for a "server". IP is perfectly fine for transfering a file from your machine to theirs. When you want to talk to someone over the net, there is no need for a "server". IP is perfectly fine for transmitting voice calls between your machine to theirs.
Your mistake is in your assumption that you even need a server in the first place. For some things, that might be useful. For other things, that is only needed as a workaround for NAT in the first place.
Also, reliably running a server at home isn't hat hard either, even today. With hardware offerings that are a better fit, it could be even easier. There isn't really any reason why hosting your own "server" at home needs to be any more difficult than hosting your own vacuum cleaner.
> Hiding insecurity is perfectly valid. It is making attack surface smaller.
No, it doesn't. It simply makes it harder for you to notice that you are not secure, that's all. This is not about whether firewalling insecure services off from public access makes the attack surface smaller. It does. But NAT doesn't, a firewall does. If you have a firewall, you don't need NAT. If you don't have a firewall, NAT won't protect you.
> I do not get pings of death, constant scanning, login attempts all the time on my local machine which is always behind NAT. Every server that has public IP gets scanned or tried out with vulnerabilities.
Which is just completely irrelevant. None of these things are a security risk. They are annoyances when trying to debug the network, that's all. And none of that is in any way fundamentally helped by even a firewall. You have a huge attack surface in your web browser that is completely unaffected by your firewall and by NAT as well, pretending that a service listening on a port is somehow a huge security problem, but executing untrusted code inside a massively complicated virtual machine is harmless is just completely focusing on the wrong problem. Also, all those pages that you load into your browser sort-of have access to your local network anyway, because your browser is inside your firewall and can connect to all those services that you pretend your NAT protects.
> I can connect totally new PC to router with NAT and not be owned in matters of minutes by some botnet.
You are constantly confusing firewalls and NAT. That is done by a stateful firewall, not by a NAT.
> My router might be exposed but it is something I know. All machines behind router are perfectly fine for remote vulnerabilities.
We are talking about IPv6 and possibilities to directly access machine where some vulnerable service might be exposed by misconfiguration. If you have remote code execution vulnerability service listening in that service it is really bad. Even pro people forget to close their database on servers sometimes, cannot think what weird stuff might be running on normal users machines.
I did not even touched running untrusted code by user because that is not in the scope of discussion. It is insecure with whatever the network configuration will be.
I do not know how you can connect to device behind NAT without setting up tunnel to it. But I might be wrong, point me to some resource please?
> We are talking about IPv6 and possibilities to directly access machine where some vulnerable service might be exposed by misconfiguration.
That is no different than with IPv4. If you have a stateful firewall, that isn't possible. If you don't, it is.
> Even pro people forget to close their database on servers sometimes, cannot think what weird stuff might be running on normal users machines.
Which is why you should have a stateful firewall. A NAT does not add anything to that.
> I did not even touched running untrusted code by user because that is not in the scope of discussion. It is insecure with whatever the network configuration will be.
It is very much in scope of the discussion, as every single end user does it. No matter how great their firewall is, you just send them a link to a website, and that website now gets to execute Javascript code on the inside of the firewall, with more or less direct access to all the insecure services supposedly protected by the firewall. Including even stuff only listening on localhost, which wouldn't be reachable directly even without a firewall. If you want to do a mass-scale attack, you serve that code through an advertising network.
So, you actually have to secure the services anyway, even a firewall is insufficient to protect vulnerable services on end-user networks.
> I do not know how you can connect to device behind NAT without setting up tunnel to it. But I might be wrong, point me to some resource please?
By sending a packet addressed directly to the internal address, which your ISP can do, anyone who compromises your ISP's edge router can do, and more often than not your neighbours can do when your ISP fails to properly isolate customers on layer 2.
NAT was (and is) destructive and painful that some of us gave up writing network software in the late-90s/early-00s. I personally abandoned several network-focused projects in the early 2000s.
The current status quo only seems "not quite painful enough" if you accept that most people cannot use true network software, limited to client-server architecture where party lines[1] communicate with each other only with the permission of central privileged imprimatur[2].
[1] https://en.wikipedia.org/wiki/Party_line_%28telephony%29
[2] https://www.fourmilab.ch/documents/digital-imprimatur/