[brynb]

Creator of Redwood here, which is similar in many ways to this and GunDB (hi Mark!). Agreed that we need more projects like this.

Redwood might be better described as a "p2p application platform" than a realtime database, but there's plenty of overlap. It's written in Go, supports HTTPS, Libp2p, and WebRTC transports (and soon, a Cap'n Proto RPC interface). It can also:

- serve as a Git remote

- stand in as an application server (you can upload assets directly to it, and they're served over regular HTTPS)

- execute Lambda-like functions written in Go, Lua, and Javascript that can be installed at any point in a given state tree

Would love some early feedback, and also to discuss similarities and differences with Sapphire's model. I've set up some demos in the "scenarios" folder -- a chat app, a realtime document editor, and a Git integration. Take a look if you're interested:

https://github.com/brynbellomy/redwood

[anderspitman]

> and soon, a Cap'n Proto RPC interface

This is interesting, and good to hear. Are there still a lot of projects adding capnproto support? It's a great protocol but I've worried it might be dying.

[brynb]

Dying? Hm, not to my knowledge. In fact, I saw a highly-commented thread on HN just a couple of weeks ago about Cap'n Proto. What gives you the sense that it's on its way out?

[kodablah]

At the least, it may be reasonable to also consider a gRPC iface (if you don't already have it).

[anderspitman]

Sandstorm Oasis going away and the last time I checked (maybe a year ago?) work seemed to have stopped on getting the protocol implemented in the browser.

[kentonv]

Hi, I'm the author of Cap'n Proto, and co-founder of Sandstorm.io. When Sandstorm failed, I went to Cloudflare where I'm leading the Cloudflare Workers project. We're using Cap'n Proto (including RPC) pretty heavily there!

That said, it's true that Cap'n Proto relies on its users to contribute improvements to the implementation. We don't currently have a team who can dedicate their time to building out capnp implementations in every language.

At present, within Cloudflare, the heaviest use is C++, with some stuff in Go, Rust, Java, and Lua. Hence, if you look at the C++ implementation you'll see it's been pretty active. At the moment we don't have a JavaScript use case so haven't put effort into that implementation. But given that Workers itself is a JavaScript platform, it seems reasonably likely that we'll end up putting some work into the JavaScript/TypeScript implementation at some point down the road.

[anderspitman]

Glad to hear it! It really would be a shame for such a great protocol to go unused.

That's interesting that you feel Sandstorm failed. I draw inspiration from it on a regular basis for several of my projects. Maybe it was just ahead of its time.

[kentonv]

Thanks for the kind words.

Sandstorm the company definitely failed (ran out of money and couldn't raise more, while having negligible actual revenue). Sandstorm the open source project continues and has actually seen an uptick in development activity lately.

[brynb]

Just wanted to thank you for your brilliant work!

[kodablah]

> Would love some early feedback

Nice project! I had considered building something similar (actually just services discoverable via DHT, nothing concerning storage). Can you discuss here or in the README its resiliency toward bad actors? Specifically, how does it work in a p2p environment with untrusted clients? If it's still an unknown, just a note on whether it's even a threat model you're developing within would be helpful.

[brynb]

Sure! Nailing down a robust security model is a major goal for Redwood. There are currently a few different mechanisms, some already partially built, and others still in the research stage.

For what it's worth, I have a silly side project goal of reimplementing Bitcoin on top of Redwood, just to prove it can provide the same (general) security guarantees. Not sure I'll get around to that any time soon, though :)

Anyway, here's what we do:

1. Users are identified by a public/private keypair (actually using Ethereum's implementation at the moment, just so I could focus on other stuff). All transactions must be signed by the sender, allowing recipients to verify the sender identities.

2. You can place "transaction validators" at any node in your state trees, and any transaction affecting the subtree under the validator will be checked by that validator. Currently, there's a "permissions" validator included that gives a simple way to control writes based on the Ethereum keypair I mentioned above. The current implementation of the permissions validator isn't very well fleshed out yet -- it just lets you specify a true/false write permission associated with a [user, keypath regex] pair -- but it's a decent placeholder for now, until I can get around to roles and policies.

3. You can also write custom transaction validators in Go/Lua/Javascript, which should make it trivial to implement just about any access control model you desire.

4. You can also create a "private" tree by explicitly specifying the set of users who are allowed to read from and write to that tree. The default Redwood node implementation does automatic peer discovery and keeps a list of peers whose identities/addresses have been verified. When it receives a transaction for a private tree, it only gossips that transaction to the tree's members (as opposed to its behavior with public trees, which is to gossip transactions to any peer who subscribes to that tree).

5. We also want secure persistent storage for transactions so that we can get high availability and redundancy without compromising the security model. To facilitate this, we allow you to configure the node to talk to what we're calling a "remote blind store" (essentially a key-value DB running on a separate piece of hardware, possibly off-site). The Redwood node encrypts transactions with a key the remote store doesn't possess and sends it over gRPC to the blind store.

6. Some applications will have a high volume of transaction data, and will want to be able to prune/merge/rebase that data to save space. To prevent bad actors from issuing malicious prune requests, the remote blind stores are going to implement their own p2p protocol involving threshold signatures. Prune requests will only be honored if a quorum of these blind stores sign off on them.

We've been thinking about this quite a bit, but not publicly. I'd love for someone to help us punch holes in it!

[marknadal]

Nice work! :)