Hacker News new | ask | show | jobs
by soumyadeb 2232 days ago
>Many of our customers use us for their ETL pipelines, they appreciate the ease of use of the platform and the performance boost from automated tuning.

This is quite interesting. Founder of RudderStack here (we are a CDI or simply an open-source Segment equivalent). I have seen a similar pain point across some of our customers. They use RudderStack to get data into S3 (or equivalent) and then run some kind of post-processing Spark jobs for analytics/machine-learning use cases. Managing two setups (RudderStack on Kubernetes + Spark) is a pain.

A singly managed solution with Spark on Kubernetes makes so much sense. Would love to figure out how to integrate with you guys.

1 comments

Congrats for RudderStack, what you're saying makes a lot of sense. Reaching out to you directly to follow up on a potential integration!
Thanks a lot. Will follow up with you.