[vasili111]

>you'll need good OO principles

Could you please give more details why this is important? I have good experience with dealing with data, data science and little bit of data engineer too but I never saw the necessity for OO. I'm also very interested in data engineering and was wandering why you mentioned OO and why it is important for data engineering?

Thank you.

[claytonjy]

I've been there too, especially since most of my pre-eng work was in R. In python at least, if you want to write code that others can use and extend, you need to embrace objects. Nothing fancy necessarily, but you should have a sense of how to organize a class hierarchy for a given program, when and how much inheritance to use. To do that you'll touch on a lot of small things like mixins, MRO, ABCs, etc.

One way you may be forced into this is custom Airflow operators; the community now recommendeds writing ~0 logic in airflow and sticking it all in docker instead, but any team using airflow for more than a few years has a tangled web of custom bullshit you'll be expected to maintain and extend.

You can certainly write a lot of python in a more procedural and/or functional way, but if you ask a python engineer to use or modify that code, don't be surprised by their anger.

[vasili111]

Thank you so much for your answer!

I have general understanding of OO in Python. I just do not see where exactly to use it in data engineering. Could you please recommend any book/article/video that shows with the examples when to use OO in data engineering tasks?