[thecolorgreen]

I'm a SWE and data engineering actually sounds super interesting to me. Unfortunately, my day-to-day doesn't provide opportunities to work with the massive amounts of data we generate. I've looked into learning this stuff online but courses like DataCamp seem too basic (I have experience with Python and data cleaning in a research setting along with some academic ML experience) or downright a bit scammy. Many of the articles I read online about this also frame data engineering as a way to transition into the tech industry, which isn't my blocker. Does anyone have advice to help me transition away from pure software to a job as a data engineer?

[nojito]

We just hired a SWE turned Data Engineer. You don't need to handle massive data to make the transition. I believe all the person did was build a small but robust pipeline that took some API response data, cleaned it, populated some sqlite dbs, replicated it for a small team to use and kept it updated every few days automatically.

[thecolorgreen]

That's good news for me. For a minute, I genuinely thought I'd have to waste time going through a bunch of stuff I already knew to learn a tiny bit and earn a certificate to prove my skills.