[hatware]

Agreed. The author is blaming Airflow for what are ultimately poor architecture decisions.

I will admit it's not easy to figure out best practices with Airflow, but if you make bad decisions and your system doesn't scale with the problem, you didn't understand the problem or how to solve it in the first place. The tools you chose are second to that.

[peteradio]

You may not know very precisely the time constants you are dealing with in your problem until you give it a shot.