Hacker News new | ask | show | jobs
by jcnnghm 1190 days ago
Is there an underlying stream processor (e.g. Flink)? How many messages per second can it process?
1 comments
calebtv 1190 days ago
All of our processing is done via Ray (https://www.ray.io/). Our early benchmarks are about 5k mesesages per second on a single 4 core VM, but we believe we can increase the with some more optimizations.

This bench mark was consuming a Google Cloud Pub/Sub stream and outputting to BigQuery.

link
dmatrixjsd 1189 days ago
Delighted to hear your choice of Ray and building atop Ray.
link