Hacker News new | ask | show | jobs
Building a Scalable Postgres Metrics Backend Using the Citus Extension (citusdata.com)
96 points by mmcgrana 3582 days ago
5 comments

If you haven't already seen it, Dan from Heap Analytics has a great talk about how they use Postgres and Citus.

https://www.youtube.com/watch?v=NVl9_6J1G60

We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before.

On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries).

If you have a lot of data and know Postgres, give it a chance.

[0] https://pexe.so

> On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries).

Is that 20K separate transactions or are some of the 20K inserts grouped together?

Separate (going through the master).
This is a super cool product. Good job.
Interesting product - good luck :)
If you're looking for a metrics backend built on traditional Postgres, check out Tgres.

http://grisha.org/blog/2016/07/29/state-of-tgres-2016/

+1 for tgres https://github.com/tgres/tgres On an aside, I found his posts on holt-winters very informative.
For anyone with questions about how to properly shard a table, how to fit a problem to a distribution strategy, or general questions about deployment, Citus has a Slack channel which is active and extremely welcoming. You can request an invite here: https://slack.citusdata.com/

Disclaimer: don't work there, but on the Slack and the Citus team is nothing short of awesome.

I had a look at Citus when evaluating time series databases and the documents said that work was still under way on masterless setup for faster ingestion (up to 500k metrics /sec).

https://docs.citusdata.com/en/v5.2/performance/scaling_data_...

If that has changed I'd add it to my table.

https://docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCE...