[mickeyp]

Airflow does not do the work itself; you write stuff in Python, so you _could_ make it do it, but it would be the wrong way forward for large volumes of data if time is of the essence. It merely calls out to stuff that does --- such as other tools that do the processing, and so forth.

One example is perhaps a small python script (run by airflow) to pull the files you need to process; pass them to in a downstream task that runs a shell script; which takes its output and in turn does something else entirely.