[motorest]

> I was surprised because Cloudflare’s central messaging is that their network is fast, and disappointed becuase I’m a happy user of their other products (domains, DNS, pages, and R2).

I've glanced through D1's docs and I immediately noticed system traits like:

- database stored in a single primary region where all writes need to go,

- cold starts involve opening a connection to D1,

- cache misses in local replicas involve fetching data back from the primary region,

- D1 is built upon sqlite, which I think doesn't support write concurrency well.

- D1 doesn't actively cache results from the primary region to the edge, so you'll have cache misses pretty frequently.

Etc.

These traits don't scream performance.

[auszeph]

My take from reading some docs is that you've got to partition your data properly, likely per-user. Then hopefully most of that users interactions are within the same datacentre.

[vlovich123]

That’s what the docs say but if you try to do this you quickly realize that the docs are living in a pipe dream. It’s not possible to set up per-user data in D1. Like in theory you probably could, but the DX infrastructure to make it possible is non-existent - you have to explicitly bind each database into your worker. At best you could try to manually shard data but that has a lot of drawbacks. Or maybe have the worker republish itself whenever a new user is registered? That seems super dangerous and unlikely to work in a concurrent fashion without something like a DO to synchronize everything (you don’t want to publish multiple workers at once with disjoint bindings & you probably want to batch updates to the worker).

When I asked on Discord, someone from Cloudflare confirmed that DO is indeed the only way to do tenancy-based sharding (you give the DO a name to obtain a handle to the specific DO instance to talk to), but the DX experience between DO and D1 is quite stark; D1 has better in DX in many ways but can’t scale, DO can scale but has terrible DX.

[manter]

I'm a big fan of Cloudflare's offerings in general, including D1 (despite the fact it admittedly has flaws).

That being said, the pipe dream statement is accurate IMO. I do think they'll get there, but like you said - a lot of the ideas put forward around a DB per customer just aren't present at all in the DX.

If I were to hazard a guess, Durable Objects will slowly gain many of the features and DX of regular workers. Once they reach parity workers will begin end-of-life (though I'd guess the workers name will be kept, since 'Durable Objects' is terribly named IMO).

This is kind of what happened (is happening) with pages right now. Workers gained pretty much all of their features and are now the recommended way to deliver static sites too.

For me, this can't come quickly enough. The DX of workers with the capability of DO is game changing and one of the most unique cloud offerings around.

It'll bring a few new challenges - getting visibility across that many databases for example, but it completely removes a pretty big chunk of scaling pain.

[Spunkie]

You may be interested in this new CF announcement, D1 read replicas and D1 sessions: https://blog.cloudflare.com/d1-read-replication-beta/

[motorest]

> My take from reading some docs is that you've got to partition your data properly, likely per-user.

I dont't think your take makes sense. I'll explain why.

Cloudflare's doc on D1's service limits states that paid plans have a hard limit on 50k databases per paid account. That's roomy for sharding, but you still end up with a database service that is hosted in a single data center whose clients are served from one of the >300 data centers, and whose cache misses still require pulling data from the primary region. Hypothetically sharding does buy you less write contention, but even in read-heavy applications you still end up with all >300 data centers having to pull data from the primary region whenever a single worker does a write.

[Spunkie]

You may be interested in this new announcement about D1, https://blog.cloudflare.com/d1-read-replication-beta/

Read replicas for lower latency and sessions for partitions per user like you mention.

[rxliuli]

It looks interesting, I will see what effect it has.