[zaptheimpaler]

Somewhat outdated view. This may be the current stack, but its outdated now and is slowly being replaced. The new view is not big data pipelines and ETL jobs, its lambda architecture, live aggregations/materialized views and simple SQL queries on large data warehouses that hide the underlying details. The batch model may still apply to ML I guess, but I'm no expert there.

[tomnipotent]

This is true for only a very limited subset of data producers that need real-time or near real-time data included in ML models. For 99% of the rest, batch processing is just fine and considerably more economical.

[molsongolden]

Any resources/guides you'd recommend?