[jshen]

It is admittedly an oversimplification for affect, but it’s more or less what you said. Maybe I’ve worked at bigger companies, we have over 1000 devs in my division, and trying to get them all aligned is very very difficult. I think central planning is a good analogy for that. The architecture that requires less central planning/coordination will likely do better in such a dynamic.

[mrdoops]

I expect streaming will be adopted like anything - first in smaller more mobile teams working in greenfield contexts, then later in large organizations when the idea is less novel and lower risk/cost to apply at scale.

Regardless I feel streaming has been around long enough now it should be urgently considered anywhere where the words "data pipeline" and "scale" are thrown around together frequently.

[jshen]

I’ve implemented it several times, and it hasn’t been clearly better. Don’t get me wrong, the alternatives haven’t been great either. I think it’s just hard when you have large organizations, large number of systems some of which are decades old, and you have high volumes of interactions/transactions.