Hacker News new | ask | show | jobs
by evanweaver 3378 days ago
Fair enough; I think it depends where you draw the line between key/value store and database.

Both of those depend on other distributed storage systems under the hood, as far as I am aware? Or is Datastore an end to end system? I know Firebase was backed by MongoDB.
2 comments
wsh91 3378 days ago
Datastore runs on top of Megastore [1]. You can find out more about our data model here [2], but it's definitely not limited to key-value data.

Our end users don't have to think much about our storage system, though, if we're doing our jobs right. :)

[1] https://cloud.google.com/datastore/docs/articles/balancing-s... [2] https://cloud.google.com/datastore/docs/concepts/entities

link
elvinyung 3378 days ago
An interesting observation that I can't seem to "un-observe" is that Megastore is actually a lot more like MongoDB than one would expect.

Both ostensibly work best when the application fits a hierarchical data model (entity groups vs. documents), and provide out-of-the-box strongly-consistent transactions for a single entity group. MongoDB feels like schemaless Megastore.

link
wsh91 3378 days ago
:)

Here's a paper with which you might already be familiar, but it's one of the citations for the Megastore paper: http://adrianmarriott.net/logosroot/papers/LifeBeyondTxns.pd.... You'll probably enjoy it (if you haven't already!).

link
elvinyung 3378 days ago
If I remember correctly, Datastore is basically a thin layer on top of Megastore[1] (aka the precursor to Spanner).

1: https://static.googleusercontent.com/media/research.google.c...

link
itcmcgrath 3378 days ago
It's a lot more than a thin layer, but like most systems at Google is a specialized layer that utilizes built on top of more fundamental building blocks that are designed and proven to do a particular job really well.

In this case Megastore provides the underly multi-region/datacenter K-V replication services.

All the database features like secondary & composite indexes, query language, multi-tenancy support, PAYG model, etc, etc, are built in the Cloud Datastore layer.

link
elvinyung 3378 days ago
> secondary & composite indexes

Interesting, so you don't use Megastore's indexes?

link
wsh91 3378 days ago
As you noted earlier, Megastore has a schema and we don't, so we have our own index implementation, yes. :)
link