[dominotw]

Same! My dataengineering team wants everything to be moved from batch oriented to 'stream oriented'. We are replacing our batch workflows to flink/spark streaming. But no one has been able to answer me what business use cases need streaming or what if any problem is being solved by this added level of complexity.

[disgruntledphd2]

Sometimes, streaming makes sense. When I was at a FAANG, we had a sampled streaming database with nice plots (kinda, ugly but functional).

This was incredibly useful as it allowed teams to figure out what had happened in prod without needing to wait for the next day's batch jobs.

OTOH, most companies don't need something this complicated, and I'd imagine you could do the same with MySQL/Postgres if you're not at FAANG scale.

[Swinx43]

I agree. When it is the right solution to the problem at hand there is nothing better than a good streaming solution.

It goes for pretty much all technology, that you need to choose the right tool for the job. For some reason the marketing just gets most people all worked up and ready to fit problems into technology that was never designed for their problems.