Hacker News new | ask | show | jobs
by masukomi 3784 days ago
is it me or is this going to result in massively long data files that you KEEP having to re-downloading in their entirety every time you want to get an update from someone you follow?
3 comments

This could be solved by archiving/rotating old entries, and have the webserver return the n latest.

For instance, you could let nginx do this:

  location /masukomi.txt {
     default_type 'text/plain';
     content_by_lua 'local tail = io.popen("tail -n 20 ".."/chroot/twtxt/masukomi.txt", "r")
     for line in tail:lines() do
       print(c, line)
     end
     ';
  }
The problem is that if you didn't fetch the file at the right time, you will lose some history after rotation happens. At the very least, it needs a standard way to deal with that (like a mandated date-based URL format?), which has to be specified in the format.

Also, no DM and no @-replies. For those, you need a registry of some sort, but decentralized registries enable name-squatting.

Before you know it, you have reinvented the WWW, except limited to 140 chars.

> if you didn't fetch the file at the right time, you will lose some history after rotation

Very true, and that's a trade-off, not unlike what Twitter does today. In my pseudo-code I didn't make the -n part of the url (it easily could be, but that'd add changes to the client). So I agree with your criticism, but a "since" lookup would be a fairly easy extension of the client (i.e. part of query, so unsupported frontends still work), as the data file already contains date.

> Before you know it, you have reinvented the WWW

Yes. :)

No need for all this complication.

Simply use IPFS[1] instead as the underlying protocol.

[1] https://ipfs.io/

Yup that's an issue. Their should be more control on how the data is stored. I like the concept tho.
Can't you just use an HTTP range request?
Yes, should be possible. But then again you would have to specify an ordering, i.e. are newer status updates at the top or at the bottom.
It can help to avoid re-downloading a timeline that didn’t change but you’ll still have to re-download the whole file for each new status.
Make the ETAG the size, send a range request for only the bytes you haven't seen already?
This is nice as long as nobody changes old status updates. Thought about using If-Modified-Since to at least reduce the load when nothing has changed at all.

But still: It will definitely take some time till a "twtxt timeline" causes more traffic than browsing the current Twitter homepage. :3