Hacker News new | ask | show | jobs
by sails 1807 days ago
IMO data engineer roles are further subset into:

1. kafka / streaming oriented software engineering

2. data warehouse and ETL/ELT development for analytics

1 comments

A good data engineer understands and can work with both of these.

They're both "data in, data out" mental models that are part of the Lambda architecture which every data engineer should at least know about [0].

But if you want a specialist streaming person to optimise all the streaming pipelines, then sure hire a specialist.

[0]: https://en.m.wikipedia.org/wiki/Lambda_architecture