Hacker News new | ask | show | jobs
by chatmasta 1895 days ago
I have a pet idea for how to make a killer news reader. I'll never build it, and I'm working on a different startup now, so feel free to take it (tell me if you do because I'd love to use it!).

The idea is you let people code open source scrapers in JavaScript for each data source (or data source type). Developers just need to extend a base class and implement an interface with methods for e.g. getting the latest posts from this data source, fetching a single post for this data source, etc. The SDK would include a toolkit for things like stripping ads / getting the meat of the content. It would also include renderers for different data types, e.g. video vs. article. But you could maybe let the developer specify a custom renderer too, if you get the whole thing working in some sort of web shell.

Then you let users subscribe to arbitrary data sources, and you let them package them into playlists (incidentally, another comment here suggests the same -- probably a good sign). Users can subscribe to other users' playlists. For choosing data sources, users can select one from a list of pre-approved sources, or specify a GitHub URL to a scraper implementation.

When the user opens the app, you call the e.g. `fetchUpdates` function on all the data sources they're subscribed to. All the fetching happens on the client, which avoids IP blocks / bot detection / DMCA SPOF. (Alternatively, you could execute the scrapers on a server somewhere, and have the client open a proxy that the server can use when fetching content for that client.)

Other than that, users browse content in a fairly standard interface. I'd look at Apollo Reddit client as the best-in-class for content browsing on iOS.

Originally I thought it would be cool to build the main core of the app in Go (or Rust, or anything cross-platform), and use a JS engine to implement the scripting. Then you could make the shell in React Native and most of the platform-specific code would be bridging to the Go/Rust core. Unfortunately, I think Apple would have a problem with running arbitrary JS code. You can only run untrusted code in JavaScriptCore. So you would need to find a cross platform way of running sandboxed JS, using JavaScriptCore on iOS and some equivalent on Android (idk the ecosystem). (The first time I was thinking of this app was a few years ago, and this is actually how I discovered Ryan Dahl working on Go + JS, which later became Deno and Rust + JS).

You would get pushback on this if you focus too much on fetching content / removing ads / circumventing paywalls. But the beauty of it is that all the logic is happening client side. And if, for example, you're using JavaScriptCore on iOS, how is that any different than Chrome for iOS? So to avoid the pushback, you can market the app as a custom web browser for news.

Bonus points for making a web client or open sourcing the protocol so people can do things like create a "morning PDF" for arbitrary groups of sources, each with its own fetching logic defined by a plugin.

It's kind of like a combo of Gopher and Greasemonkey.

If anyone seriously wants to build this, email me and I'd be happy to talk through the idea, you can have it 100%, I just think it'd be awesome. Would also be tons of work to get right.