Hacker News new | ask | show | jobs
by bsder 923 days ago
Is there a reason why you can't run your own Usenet server nowadays?

In the old days, the issues were that bandwidth and disk were expensive. That's no longer true.

Is there a techncial reason we can't have individuals running NNTP nodes?

4 comments

I think you can run Leafnode if you want to host your own, small groups. Otherwise (my understanding) you need to peer with other providers and be able to consume something like 30-100TB/day if you want all groups on the network (unless you can find providers to peer with that let you limit groups)

Some discussion here https://www.reddit.com/r/usenet/comments/ziqdh2/become_my_ow...

Note that 30-100TB/day is mostly copies of pirated copies of movies, getting re-encrypted each time so you have to pay someone to get the encryption key and to avoid detection.

If you want text groups, it must be a lot lower.

Text content is under 100MB per day, including spam, according to some Usenet admins.
I imagine so but it sounds like you're still at the mercy of a provider to peer with you (which has a cost to the provider, even if low)

I think that's where the "build your own network using nntp" comes in

Well, at this point the mainstream usenet network is dedicated to piracy. That's what it is - 100% piracy. The only reason they use NNTP instead of something like BitTorrent and call them "articles" instead of "files" is to maintain plausible deniability that that isn't what they're doing.

The textual discussion network may as well be its own wholly separate network. I only know of Eternal September, but I'm sure there are other servers.

Correction: there are several other servers and the text part is less dead than I thought.
If Usenet's requirements haven't increased in a 2-3 decades, and in its moribund state I expect they haven't, then it should be trivial to run Usenet servers on almost any device, at least in terms of resources.
Guess it's hard to get others sync with you?
The principle use-case for Usenet has always been small, controlled-access networks.

That's what ARPANET and the early Internet were, where "small" meant < 1 million active participants and "controlled access" meant meeting the requirement of belonging to one of a limited number of academic institutions, tech companies, government agencies, or select military units. Total Usenet participation in April 1988 were 140,000 readers, by Eric Reid's DEC surveys, as cited in John S. Quarterman's The Matrix (1990):

<https://archive.org/details/matrixcomputerne0000quar/page/24...>

Spinning up an NNTP group for internal company communications, for a family group, or for a (sufficently tech-savvy) organisation or hobby group would be a viable option. My experience is that peak group experience tends to occur with somewhere between about 5 and maybe 1,000 participants on the high end, and that's being generous. In practice, 5--50 is probably a better sweet spot.

The technology isn't the stumbling point nearly so much as convincing people to use the service. In some industries, privacy and disclosure rules may preempt usage (e.g., healthcare, finance). And the fact that Usenet has no intrinsic authentication or attestation of identity (though these can be provided by additional mechanisms such as PGP/GPG signatures) means that publicly accessible newsgroups are likely to be nonviable given malicious actors.

What's underappreciated about much of the early Internet / online world is how small most of the "large" (as in influential) groups or movements were. Usenet, the WELL, Slashdot, and early blogging had core groups of perhaps a few hundred to a few thousand people, possibly fewer. Yes, there were additional participants in many cases, but in terms of how concentrated the bulk of activity was, small.

Hmm and if you limit the use case to those small groups, there are a lot more options now. There's the whole Fediverse stack, of course, but also lots of stand alone forum options you can self host. And why self host when there are also lots of low-cost community platform options out there?
So, that really wasn't the original question, so noting that we've moved on from that ...

Basic protocols such as NNTP and IRC have the benefit of being stable, small (running a Usenet server on an OpenWRT router with additional writeable storage would be doable, running a Fediverse instance not so much), and being relatively client-independent and even protocol-flexible. NNTP <-> SMTP gateways are, for example, A Thing, and people can participate in such newsgroups from either Usenet or email clients. Similarly Web gateways to either NNTP or SMTP systems.

The flipside is that the underlying protocols are also simple and absent some additional tooling lack much of what it is that people have come to expect from more recent messaging systems:

- Mobile device access. That's probably the big one, though I'd argue that for useful discussion you're better off eliminating mobile use. I've commented that losing a keyboard and being forced to a small screen cuts my effective IQ by half, and that quality of discussion on public systems also seems strongly negatively affected by mobile use. Read-only access might be a compromise, with some "I have something to say but will wait until I can access a Real Computer" might be an option. Android + Termux + Bluetooth keyboard meets my minimums as "real computer", FWIW.

- Formatted text, usually a flavour of Markdown or some "smart editor" for normies. Then again, we seem to survive with much less than that on HN.

- Images. That's the principle feature NNTP lacks natively, though they can be shoehorned in via uuencoded posts. Along with audio and video, the ability to create multimedia posts (popular, occasionally useful, terribly abused) might be an objection raised by many.

- Third-party clients. The great strength/weakness of NNTP and SMTP are that neither are locked to a single client, or Web client (though the latter is available). Having a specific tailored client program (tin, slrn, Emacs, pan, etc.) is useful because the features of the platform are strongly supported by those clients. Downside is Yet Another Piece of Software to install and train on. In the mobile world apps have been breaking the notion of everything-in-the-browser, though often bringing along their own set of issues, most notably surveillance, but also general issues of design, UI/UX, maintenance, security, and abandonment.

The initial question addressed could and the answer really is "trivially", from a technical level.

Yours is "should", and as I'd hinted in my initial response, that's far more social and up for debate (as we're doing here).

But at least you're demonstrating a exception to Ian Malcolm's could/should criticism.

It's actually pretty trivial. Eternal-september seems happy to peer with anyone who asks (just send an email), and there's a newsgroup specifically for asking for peers; everyone who posts there gets lots of offers pretty quick.
no. it's just a tcp client/server model like any other. your isp probably won't let you host services but that's been the case for decades
> your isp probably won't let you host services but that's been the case for decades

How would your ISP know?

NNTP originally was originally store-and-forward over dialup like links, no?

So, if I contact a different server and upload/download the changed data, how would they even know?

I guess the big issue would be having some "rendezvous" server in order to help with NAT punching as well as authentication.

No, NNTP is a TCP protocol. Early USENET moved messages over systems like UUCP.

NNTP is to USENET what ActivityPub is Mastadon. USENET is a confederation of cooperating peers, currently using NNTP, but it could be any mechanic to move the messages. You could wire it up using SCP I imagine if you were motivated to do so.

Specifically, the NNTP protocol does not talk directly to how USENET works. For example, you can find out how to exchange messages for particular topics using NNTP, but not how to actually create those topics. That left to the actual software and administrators. USENET using the concept of "Control Messages" to exchange information about things like newsgroup status, but the content and format of those messages are not specified in NNTP.

As a USENET peer, pretty much all peers are "equal". You host your own news server, you tell it what groups you're interested in, it peers with another host and exchanges messages about those groups. At that level, they're equal.

But just because you create a newsgroup on your system, doesn't mean that group is instantly propagated across the planet. That's where the USENET governance kicks in as to who is going peer and distribute new groups, or not. Each individual relationship within the peer group can be different.

I can't say exactly how a news client differs from a news server. It may not have any of the peering logic, posting and reading groups directly from a server which then handles the peering. News servers "peer", clients are lighter weight.

The protocol is from the era where each protocol used a dedicated TCP port rather than "it's all JSON over HTTPS" like now.
Yeah. Amen to that

There were some bad ideas in the beginning, FTP as an example of several of those

FTP splitting the data and control ports was a smart implementation. It allowed for optimization of the ports (one focused on throughput) and meant that control could still occur even while long responses were happening on the data port.
I haven't tested it, but I imagine it allows you to trick an FTP server into making an HTTP request (without TLS).

It probably seemed like a good idea at the time, and there was no way to know the problems without trying it.

It also allows you to send a command to cancel an ongoing transfer.

it also allowed for FXP, which was a godsend in early pirating/warez days ;-)

...or so I've heard

This could have been better accomplished by multiple connections to the same port.