[braza]

I share the same enthusiasm related to ETL as a code, but as a heavy user of SQL Server 2005 (SSIS) at that time, the main advantages for me were that there was a closed and concise ecosystem around the technology, less fragmentation and multiple pieces that could fail. It was way simpler to reach people to solve the issues on the platform, and less time was needed to think about the technology itself and to think more about the problem.

When I transitioned to ETL as a code 12 years ago, I felt for the first time what it looks like to be a programmer/developer, since in my case the technology was literally a means to an end. To be honest, I did not even think about tech at all.

Just to give an idea, I was responsible for 5 instances of SQL Server, and with another person we were responsible for the whole end-to-end data operation (i.e. ETL via SSIS + OLAP cube via SSAS + and reporting via SSRS), and today I struggle to have the same level of scale with less than 7 people.

Fast forward to 2024, the inverse is true: I spend way more time thinking about the tools and their specificities than about the problem that I am solving at the end of the day.

I'm not telling that ETL as a code is bad (actually there's a lot of good stuff out there) but for folks in this kind of ecosystem (e.g. azure), definitely the juice to have some homemade code and all the stuff that comes with that, does not worth the squeeze.