Hacker News new | ask | show | jobs
by usenetthrowa 3863 days ago
Usenet (NNTP) is a protocol somewhat similar to email. Unlike email, the default communication style is in "groups" (sort of like a mailing list for a specific topic). One posts individual messages (like one email) to a group; replies are threaded, much like mailing list discussions. Historically the groups were organized somewhat hierarchically.

The end result, with a good news reader program, is somewhat like a large global collection of topic-specific forums.

Binary posts are just a bunch of posts to the same group with a message Subject matching some pattern (usually with ascending numbers counting the posts). The content is encoded to work around the limitations of the protocol and/or provider. Historically this was "uuencode", but you can also do base64 (like email attachments) and the new jazz (hah! New in 2003, anyway) is "yEnc." yEnc relies on 8-bit safe NNTP servers and only encodes the few characters not permissible in NNTP messages (periods at the beginning of lines, I forget the rest). So you get much better encode ratios (~101-102% of original size) than with something like base64 (133% of original size).

NNTP providers, especially in binary groups, are subject to occasional message loss. So posters will typically generate Reed-Solomon forward-error-correction blocks using a program called "par2." This allows one to download binaries that are missing some messages, as well as some additional FEC (potentially on an as-needed basis), and rebuild the original content. Generating and recovering from par2 files is extremely computationally expensive.

1 comments

Follow-up:

> One posts individual messages (like one email) to a group

Messages can be posted to more than one group; this is called "cross-posting", much like with email mailing lists.

> The end result, with a good news reader program, is somewhat like a large global collection of topic-specific forums.

You can also choose to "subscribe" to a subset of the available groups; your news program is able to pull metadata only for that subset you care about. You can also download individual messages on demand, much like IMAP email access. It's fairly bandwidth efficient for clients.

follow up - unlike email, the groups are actually publicly viewable, whereas email you needed a username and password.