Hacker News new | ask | show | jobs
Optimizing Conduit – 5x the Throughput (meroxa.com)
4 points by raulb_ 461 days ago
1 comments

Meroxa improved the Conduit streaming engine by redesigning its architecture, which increased throughput by five times while simplifying the code. The new single-threaded approach processes batches of records, reducing latency and enhancing performance significantly compared to the old system. Testing showed that the new architecture can handle a peak of over 500,000 messages per second, demonstrating its efficiency with various message sizes.