[bigger_cheese]

This is something I have been grappling with for a while - I work with data in an industrial plant there is a lot of legacy data systems here and "data latency" has been a real issue - real time dashboards are seen by some subset of management as the holy grail, being able to look at any part of our plant and see what is happening in real time is extremely attractive to certain people.

For a longtime there has been a lot of resistance "real time is too hard" used to get thrown around a lot- what is really driving it in last two years or so are Machine Learning and computer vision applications. There has been a huge push to integrate ML models (for example live defect detection) into our operating process which has necessitated low latency access to real time plant data.

The data pipelines we've had to build to enable these ML applications have bough latency way down all over the place and have kind of bought other applications like real time dashboards with them for "free".

[abtinf]

I started in manufacturing with a similar view. What turned me around was reading “The Goal” by Goldratt and “Out of the Crisis” by Deming, and seeing just small applications of their principles yield significant results.

[disgruntledphd2]

Out of the Crisis (and Deming more generally) deserves to be more widely known and quoted.

It's incredible how much of what he says (especially the stuff about how production workers feel) maps directly to the technology industry.