Hacker News new | ask | show | jobs
by nchmy 57 days ago
This is really cool. I'm working on stuff that is somewhat aligned with this - offline knowledge base/educational platform focused on things like appropriate technologies for rural people in the developing world. Storing in the browser and, more importantly, searching it is definitely one of the major challenges. (it's also just a much more dynamic app)

My main question about this is whether it can be dynamically/incrementally updated within the browser? Eg new material is available or edits have been made, so sync it from backend and it gets merged in.

I've been working on using rxdb to sync and store in browser - it can use its own indexeddb abstraction, sqlite or it's own OPFS-based DB. It can also load any of these into memory in its memory-mapped mechanism. I've also made a mechanism to load everything into flexsearch in a sharedworker, so that you can do full text search fairly performantly.

It's a lot of complexity though. I'd be curious to hear any of your thoughts. Or even to chat if you're open to it!
1 comments
tnelsond4 56 days ago
I'm not sure I follow exactly, but if I understand you mean that when the database file is updated that the app updates? Right now on app load it updates the service worker and shows the files in cache first. If there's a newer file it fetches it in the background, it then sends a message to the client that there is a new file. I haven't implemented the next part yet but it should be able to invalidate the current file and load the new file without refreshing the page. Right now the new files will load the refresh after the new service worker is activated.

But the page still had to be refreshed to load the new service worker. I'm looking into ways to cut the time to loading the new files down because right now you have to refresh the page 3 times for the new files to take over.

The .peak files aren't designed to be a database that you can just add to during runtime, they're rather static and highly efficient in that context. But it's easy to edit the source files and generate a new .peak file from that.

You can take a folder of any kind of files and run peakgen on it and it will create a compressed .slab file that you can search and fetch results from just like the .peak files. I first saw that done with SQLite and I really liked it, so I knew I could do it too.

If you want to chat you can shoot me an email.

link