[neftaly]

I evaluated using this in prod, but came to the conclusion that R2 was a bit shaky as a database. CF has since come out with D1 SQL, DO (beware vendor lock-in), queues, and my favorite, MQTT pub-sub, which would be better options. They've also been teasing some vastly improved options for STUN/TURN; I wouldn't be surprised if more turnkey signalling popped up.

[vlovich123]

I don’t know what the consistency requirements are but workers Kv or r2 + cache might be smarter choices in terms of improving resilience and also performance by a lot (same goes for suggestions of using DO - R2 is kind of a weird choice to manage stateful rendezvous). Queues and pubsub feel like incorrect choices unless I misunderstood what they’re using the storage for.

[gfodor]

Author here. KV did not provide the consistency guarantees needed for this to work. If you were signaling across two edge nodes, you didn't see candidates until the data was replicated, which took a long time. R2 doesn't have this problem.

[kentonv]

This is because R2 uses Durable Objects under the hood to achieve consistency. But you'd probably get better results using Durable Objects directly.

[gfodor]

At the time I wrote this it was prohibitively expensive.

[neftaly]

R2 was used because it was the only database-like service CF offered at the time.

IIRC the P2pcf client polls the worker for new peers. Pubsub would let you use an ongoing websockets connection, plus all the channel management logic is built-in. The game-changer is that they've said they'll charge per-message as opposed to per-connection-second, unlike everyone else!

[vlovich123]

That’s not true though. DO/D2 are database as well (DO precedes R2 and D2 overlapped). KV is an eventually consistent database.

DO/D2 will likely perform better than R2 unless you’re storing everything in the metadata. KV will perform better than all of them although if the storage is to keep a list of peers the API as exists today won’t work well for that - you’ll have to do 1 key per connection and retrieve via a paged list - otherwise you can lose clients since KV has no mechanism to do durable updates of values stored within keys*. You could also do DO with background refresh of cache to maintain good performance while maintaining better bounds on how long a value stays in the cache.

* actually it is possible if you mediate updates to KV through a DO (with storage as the ground truth or without it and risk occasional lost writes if the CF infrastructure decides to kill and reset/relocate your DO).

[neftaly]

Fair enough, I didn't investigate them very far, as IMO only pubsub maps to the "webrtc signalling for misers" problem neatly enough to make it worth adopting (probably -- they haven't actually said how much it will cost).

[gfodor]

At the time I wrote this library using websockets on Cloudflare would have been prohibitively expensive compared to this polling method. Has that changed?

[neftaly]

From my understanding, yes. I asked about the billing model on the CF Discord 6-12? months ago. I don't know how much it costs per message, or if there is a catch wrt cheap wrtc signalling, however my understanding is that you won't be paying per user or per hour etc. To be clear this is through their MQTT 5.0 service only.

[kentonv]

Yes, in the last year Durable Objects introduced "hibernation", in which the server object can shut down with WebSocket connections still active, to be started back up when a new message arrives. This largely solves cost issues with WebSockets on Workers / DO.

[gfodor]

Yeah this might be a better solution now for sure.

[neftaly]

I think it'll work pretty well! I might keep something cloud portable cooking away in the background.

[gfodor]

D1 looks like it could replace R2. Most of the tricky business in p2pcf is dealing with all the ICE details - porting the persistence from R2 to D1 might not be too bad.