[jesseryoung]

Very recently implemented a "lakehouse" in Azure with Databricks and ADLS. So far the enterprise is pleased with it. Our traditional IT EDW developers like it because they can use modern software development practices (source control, CI, unit tests, etc) to build their ETL jobs. Our data analysts like it because the can get access quickly to semi-raw data (we transform it into the delta format before they can access it). As an architect I enjoy it because of how many options I have for moving data around, not to mention how easy and fast things can scale.

The biggest challenge I have is dealing with the fact that I now have an impressive amount of "python developers" doing whatever they can to solve their problems. I genuinely think we're improving the ability of our business to do analytics on top of our enterprise data, but sometimes I worry about the amount of technical debt we just allowed to accrue.