[Declanomous]

This comment will sound a bit self-serving, but it supports your point. I have most of the skills necessary to be a data engineer. My degree in biology, but I nearly got a double major in computer science with a minor in math. (I wanted to work in bioinformatics, but it's nearly impossible to make more than a pittance without a PhD) I didn't pursue the double major because I felt taking classes outside of those three fields was more useful to my development.

Instead of working as a data engineer, I'm working at a non-profit doing pretty much everything involving data for them, as well as running their appeals, and doing almost all of the analysis. I'll lead off by saying the biggest downside of working for this particular non-profit is the salary. However, there are a lot of things I like about this job:

1) Location: I want to be located in Chicago. I have 0 interest in moving out to the West Coast. I'm up in the air about working remotely, because I feel like there is a lot of value in working with people in person.

2) The role is very broad. I get to do a lot of exciting things with data, but it is also a marketing and communication role as well. I am included in nearly every strategic discussion, not just those pertaining to data or technology.

3) Work life balance is very good. I am never expected to work more than 40 hours a week. My boss makes sure that everyone is focused on their lives, to the point where he basically kicked me out of the office for a week because I was waffling about taking a vacation. He makes sure that people know they aren't expected to check their email or do work on the VPN during off-hours.

4) The work I do makes a difference. Not in a "I make something people use" difference, but in a "my work has rescued people from being homeless and fed starving kids" difference. My first couple of jobs out of college were totally lacking this aspect, and I didn't realize how much it meant to me until I started working at a place like this.

I've been here a few years now, and so it's approaching the time where I should start looking for a new job if I want to continue to grow, but I'm having trouble visualizing what that would be. From my perspective, the problem with hiring is that job listings really focus on titles rather than roles, even in smaller organizations. I think my best bet of finding an organization matches the first two points, if not all four, is through my network rather than through job postings. So, to your point, the only way I see myself in a narrow-title role like a "data engineer" is if I really need money.

[infinite8s]

> (I wanted to work in bioinformatics, but it's nearly impossible to make more than a pittance without a PhD)

Just wanted to comment on this part - sadly it's difficult to make more than a pittance even with a PhD.

[Declanomous]

Yeah, totally. The difference is a bioinformaticist with a PhD generally gets to choose what they research, whereas a bioinformaticist without a PhD has to work under someone else's grant. Biology actually has the lowest pay of any major for people working in their field with a four-year degree. You are lucky to make more than minimum wage with a four-year degree, especially if your interest is field ecology or something similar.

If you want to talk about a shortage of labor where it would matter, biology as a field is probably hurting way more for talented software engineers than any company that needs a data engineer. There are so many great applications for programming in biology, and unlike other sciences, say physics, researchers don't tend to pick up on any amount of programming skill on their way to their PhD.

I've tried getting involved in bioinformatics on the side, but it's really difficult to keep up with the field if you don't have thousands of dollars to drop on journal subscriptions. It's also really hard to get access to the data researchers use in general (in any field), but it is made even harder when dealing with researchers involving people due to concerns about privacy. I don't think a focus on privacy is a bad thing, but a lot of publicly available data is sanitized to the point where your sample size would need to be in the billions to draw any inferences. You can request access to less general data, but good luck doing that without the support of a research organization.

Anyways, unless you have a martyr complex, there really isn't any reason to go into bioinformatics.

[x0x0]

I happily worked in a wetlab writing stats software to support breast cancer research. I now do better ad targeting. My salary tripled.

[Grishnakh]

>The work I do makes a difference. Not in a "I make something people use" difference,...

I'd just be happy with that. Most of the work I've done professionally hasn't gone anywhere; it's always "we missed the market window" or "upper management decided on a new strategy". I can't point to that many things I got to work on that actually made it into the market and were used by people for long. One place (a semiconductor company) had a successful though buggy product and large customers in place, with the product already deployed into the field, and the software I wrote got used by some customers, but then suddenly the company decided they weren't making a big enough profit margin on this part (even though the profits were guaranteed and extremely low-risk as the customers had the part designed-in), so they simply quit the market and laid off our entire team.

Making something people use would be a step up. Rescuing people and feeding starving kids is a pipe dream, but then again I work on embedded devices, not big data or analytics or anything like that so that's not exactly a position that'd be easy for me to find if I really wanted it.