[yresnob]

"Finally, at query time, we bring together the real-time views from the set database and the batch views from S3 to compute the result"

so how in the heck does this work? at query time you decide what file to get our of s3 (hwo do u decide this?), parse it, filter it, and merge with the results from the custom made Redis like real time database?

[paladin314159]

The files in S3 are pre-aggregated results keyed by how we fetch them (e.g. there will be a file containing all of the users active on a particular day). What you've described is a pretty accurate description of what happens :)

[sskates]

We'll be sharing more about our query architecture in the future as well as other parts of the stack that we haven't included here. The query layer is an impressive piece of architecture that handles fast access to multiple distributed data stores.