|
|
|
|
|
by sskates
3971 days ago
|
|
We offer Redshift and raw SQL access as an add-on to the analytics, it's not used to power the dashboards at all. We've actually developed an in house backend from scratch that's gone through ~10 or so iterations. There are 3 main data stores: 1) We have a real-time in memory data store (similar to Redis) for recent data that aggregates data in various ways to display on the dashboard 2) We have a batch service backed by Amazon S3 that for data older than 24 hours that also aggregates data 3) We have a column store for more complex queries that can't be represented by the aggregated data On top of all that we have a distributed query engine that access all the data stores and queries the appropriate ones in parallel at query time on the dashboards. We'll have a blog post that has more details about our stack in the next week or so. |
|