Oh wow, very relevant indeed! I guess I thought ephemeral DBs would be better so that a user’s expensive query wouldn’t bog down the db for other users. And rather than just limiting them, enabling them to do whatever queries they could with pg running locally
The query layer is composed of ephemeral Postgres instances that get created on-the-fly. These instances either proxy to live data via FDW, or they lazily download the columnar fragments necessary to resolve a query. This is meant to perform as a horizontally scalable caching layer for shared queries. For more optimized/predictable use-cases you can “check out” a version of a data image ahead-of time. In this case you use the `sgr` command line tool to load the objects of an image into a Postgres database that you can then scale vertically as normal.
Our site is a bit out of date — we’ve got a new marketing site coming shortly, and we’ve gathered an all-star engineering team to take this to the next level. Stay tuned :)
(Or better yet if you want a unified data stack for your team, please get in touch… we can load data from 100+ sources into versioned Splitgraph images, or we can index and proxy to existing data sources with federated querying).
This is something we are currently building at Klarrio on top of YugabyteDB.
We have opted for YugabyteDB table spaces to solve the isolation problem. We allocate a number of dedicated tablet servers to a tenant so that their db objects are close to each other. YugabyteDB executes queries only on those tablet servers where the data relevant to the query is. A resource heavy tenant doesn’t have an impact on other tenants. It’s not really “serverless” but does seem to solve the problem at a cost.
Tenants receive complete databases with grant permissions so they can do as they please.
Because YugabyteDB uses Postgres 11 under the hood, we have an additional sanity layer in form of a Postgres process utility hook which prevents the tenant from escaping the sandbox.