Hacker News new | ask | show | jobs
by PeterZaitsev 1705 days ago
Thank you for your prompt response!

I think the most important thing is Clickhouse is NOT designed for small batch insertion, if you need to do 1000s of Inserts/sec you do queue in front of clickhouse. And query speed can be impacted by batch side a lot. So have you looked at query performance with optimal batch size ?
1 comments
mfreed 1705 days ago
Yep! The blog post includes data and graphs from both large (5000-15,000 rows / batch) and small (100-500 rows / batch) sizes. Please see the section "Insert Performance". Thanks!

https://blog.timescale.com/blog/what-is-clickhouse-how-does-...

link
PeterZaitsev 1705 days ago
1) This is also small batch size. If you're inserting 500.000 rows/sec 5000 rows is not particularly large batch size

2) I see different graphs for ingest but not for queries. The data layout will depend on the batch size, unless of course you did OPTIMIZE before running queries

link
ryanbooz 1705 days ago
1) you're absolutely right. 5k rows isn't "large". We also mentioned that we did hundreds of tests often going between 5k and 15k rows/batch. The overall ingest/query cycle didn't change dramatically in any of these. That is, 5k rows was within a few percentage of 10k rows. Interestingly, the benchmarks that Altinity has, only used 10k rows/sec (which we also did, it just didn't have any major impact in the grand scheme of things).

2) We did not specifically call OPTIMIZE before running queries. Again, learning from the leaders at Altinity and their published benchmarks, I don't see any references that they did either, and neither does the TSBS code appear to call it after ingest.

Happy to try both of these during our live stream next week to demonstrate and learn!

Altinity benchmark (10k rows/batch mention): https://altinity.com/blog/clickhouse-for-time-series

link