Hacker News new | ask | show | jobs
by xfax 3798 days ago
Mind if I ask how you use Spark for your ETL jobs?
1 comments

Feature engineering. Transfers about 3.5b records into features that go into a variety of models. Previously was a hadoop streaming job (~40 hours); now about 6.