[biellls]

Depends on where you work at.

A lot of the time they do ETL/ELT mostly with airflow nowadays.

Some use little more than SQL and if they're lucky something like DBT to template and schedule the queries.

A lot of SQL now runs on data warehouses of which snowflake and Google bigquery stand out, but it's SQL nevertheless.

A few lucky ones get to use Spark which is more interesting as it's written in scala or python, but from my experience 99% of the time businesses are better off training business analysts to use SQL and writing the queries in snowflake.

Another interesting area is real time data, for existing e with Kafka. This can be challenging and fulfilling except a lot of the time businesses people will tell you it's cool but they don't want the data to keep changing so could you please aggregate it every day... So you end up with an overly complicated batch system anyways.

For me the sweet spot of practicality right now is Airflow/DBT/python/snowflake.

[marcosdumay]

> A few lucky ones get to use Spark which is more interesting as it's written in scala or python

If you want a fun language, it's really the wrong place to be.

The work can be very fun, but you must take the fun from data analysis. The best software engineering can do for you is to not get on your way (and SQL excels on this).

I'd say the people working with Spike are the unlucky ones. Because they have to focus more on their tools, and not on the analysis.