Hacker News new | ask | show | jobs
by fmeringdal 1041 days ago
Not halted by any means. It currently relies on Litestream to do streaming replication, but unfortunately the author of Litestream removed streaming replication. I have been working on a Rust rewrite of Litestream which will be ready within a few weeks and will be incorporated into Covert. My focus on the Litestream rewrite is the reason there hasn't been any commits directly to the Covert repo the last months.
2 comments
benbjohnson 1041 days ago
Litestream author here. The streaming replication was moved to a different project called LiteFS[1]. That may be a fit for you or, at the very least, it could be a helpful reference for your Rust port.

[1]: https://github.com/superfly/litefs

link
fmeringdal 1041 days ago
Yeah, I haven't looked too deeply into LiteFS yet. I need the replication to work well with SQLCipher and the ability for the Covert server to completely control the master encryption key without passing it over any network or language barrier (this means I would have needed a Rust rewrite of Litestream even if it still supported streaming replication). Thanks a lot for releasing Litestream to the public. I have learned a ton about SQLite from reading the codebase!
link
benbjohnson 1041 days ago
That makes sense! Let me know if you have any questions. There's a Litestream Slack you can find on the home page or email me at benbjohnson@yahoo.com
link
the_duke 1041 days ago
Very cool, I'd have a big use case for "Litestream in Rust" as a library.

Any way to get notified about the release?

Also: will this be a straight port, or could it also support "live-replication" from S3, instead of just on-demand restores? https://github.com/backtrace-labs/verneuil supports this. Sadly it isn't very actively maintained and buggy.

link
fmeringdal 1041 days ago
Interesting! You could follow me on Github to get notified when the repo is made public. Other than that I will probably do a Show HN and post it to Reddit as well.

It will mostly be a straight port as I think that is the optimal way to rewrite a codebase from one language to another. First step is to do an almost line by line copy paste from Go to Rust and then port the test suite and get that passing. At this point I would do a major refactoring where I can rely on the test suite to make sure I don't break anything and ensure the codebase is idiomatic Rust. Once that is done we can start extending the project with additional features.

Live replication from S3 sounds cool so please open a more detailed GH issue to describe the use-case for it once the repo is public!

link