[chimtim]

Low level programmer here. This is great but the job market for low level programmers is very small. Recently, while interviewing, I described a thread safe, concurrent queue implementation to a hiring manager and he asked me -- "so how does this relate to big data and ML"?

[fritzo]

What was your answer to the hiring manager? How about:

"[buzzword] systems need lots of training data. To process that data quickly, systems must parallelize and distribute computation. Concurrent queues are a basic building block in parallel and distributed systems. They are used both internally in [buzzword] systems and directly by engineers building ingest pipelines and serving systems. Engineers need to know the trade-offs of different concurrent queue implementations to avoid introducing unnecessary bottlenecks in [buzzword] systems."

[cookiecaper]

Yeah, unfortunately there are people behind the scenes pumping out a whole lot of BS and aggressively manipulating the dev ecosystem with VC dollars.

You have to wonder what percentage of devs are jumping on these trains out of naivete and what percentage are jumping on out of cynicism, as both are good explanations. The explanation that rarely applies is "this is the best way to solve our problem".

None of us should be so naive to believe that the developer market, or even subsets of the developer market, are a cozy little corner for engineers and tinkerers where rationality wins the day. It's big money (even if you don't pay anything for your tools), and that means that advanced marketing tactics are at work. Hackers are not invincible, though they like to think they are.

[bogomipz]

>"Yeah, unfortunately there are people behind the scenes pumping out a whole lot of BS and aggressively manipulating the dev ecosystem with VC dollars"

Can you elaborate on this? This intrigued me but I didn't follow your arguments after. Can you give a concrete or specific example?

[fnayr]

Yeah OP seems to be indicating that VC or some other actor is influencing the developer "market" to deflate salaries by inflating the supply of devs, at least that's what I'm reading it as, and I'd be very curious to learn more.

[cookiecaper]

I wasn't referring to the wage market, I was referring to the market for developer tools.

However, VCs do deviously manipulate the wage market in early startups by pumping a false narrative about how it takes young (read: naive, as these are virtual synonyms) people to innovate, and how when the company gets really super duper big, the equity will make up for the crappy salary. This trick is used to get young people to accept an apartment split with 6 other "founders" and some ramen, or in the case of early employees, a salary 40%-60% below market, in exchange for a pull on the VC slot machine.

[bogomipz]

I think the commenter two above was able to flesh out the meaning of your original comment for me. I do agree with you on the wage market though as well. I was thinking about this the the last time(recently) a recruiter was pitching the whole "equity in place of cash comp" to me.

If you figure a 1 year cliff and then 25 percent a year vesting schedule. You are into the startup for 5 years to be fully vested. How many of those can you do in a career? How many companies continue to be awesome enough to stay for 5 years? How many of those will actually IPO? The odds don't seem that great.

[code_sloth]

I doubt there's any intentional manipulation going on behind the scenes. It's just VCs throwing their money at the potential next big thing, which is almost never at something low level. That's where the money is, and the tooling + community follows. The rest is just network effect.

Just a few years ago, jquery was huge, and many companies were just starting to hop on the web bandwagon. Many frontend developers who didn't even know javascript could get better paying jobs than low level programmers. I thought that was a sad state of affairs, but that was just where the market was at.

[bogomipz]

Understood, thanks for the insight.

[cookiecaper]

Behind the scenes activity is rarely so overt that one can point to "concrete" examples, and there's a lot of plausible deniability. Surely many people are just trying to cash in on the fad (cynics as above), and they hire developers or engineers (more likely naive than cynical), and both parties end up throwing fuel on the fire, as they're both now invested in it both monetarily and emotionally. These people are, partially unwittingly, doing most of the dirty work.

I think the zeitgeist is manipulated by large players like Google and Amazon to promote their cloud offerings. They know everyone is going to follow them wherever they go head first. Thus, they focus on driving the industry toward rented hardware that they can sell at a large markup, and gives them many other ancillary advantages, not the least of which is making large swaths of the internet's infrastructure dependent on themselves, a very powerful position to be in.

There are even articles that freely admit Google released Kubernetes to get more people on its cloud [0], and they run industry groups like the CNCF ("Cloud Native Computing Foundation") to promote the idea that running your own hardware is the devil, and why don't you just let Google handle it all for you, OK sweetheart? You wouldn't want people to call you a philistine, would you? Google is cutting-edge, and you're just a working slob.

Nevermind that a lot of people are adopting k8s only because cloud servers are so expensive, and they're looking for a way to consolidate that expense. Amazon should be trying to counteract Google here and stress the high labor costs associated with running a k8s cluster and the pre-requisite conversion of apps to work well on one.

Most people don't need something like k8s and, for that matter, most people don't need something like TensorFlow. Google releases these things because they have one big positive net effect: a lot of people paying a lot of money for cloud servers.

There is no reason to run a real database like PgSQL on Kubernetes. None. If you want to do this, you are a victim.

VCs are looking for a similar type of influence, just on a smaller scale, by getting into dev tools. Dev tools are the key to platform dominance; they are, after all, the root from which the software which will keep people on your platform arises.

Microsoft understood this, which is why they were crapping themselves when Java picked up steam in the mid-90s, why they had to supplant NetScape, and why Steve Ballmer had a conniption yelling "DEVELOPERS!" not too long after that [1]. It's why they've continued to pour billions and billions of dollars into .NET and Visual Studio down to this day.

It's also why they're making moves into new platforms like TypeScript and hiring away major k8s contributors: they hired k8s co-founder Brendan Burns [2] and their acquisition of Deis was announced just this week [3]. They want to retain as much platform influence/control as possible.

Once you have the developer, and by extension the user, on your platform, there are many ways to trap them and make them give you money. Lock-in has always been the holy grail in software because it's the best way to make money. Lock-in is gained by platform control.

We need to watch our butts here and be careful about what we're willing to believe. There are many vultures looking to get a piece; I've seen this amp up ridiculously over the last 10 years, and I don't think we're even at critical mass.

---

[0] https://www.wired.com/2015/06/google-kubernetes-says-future-... ; Google's PR has been meddling here, the title used to read "Google open-sourced Kubernetes to boost its cloud", see https://news.ycombinator.com/item?id=9693853 (from 672 days ago)

[1] https://www.youtube.com/watch?v=Vhh_GeBPOhs

[2] http://www.crn.com/news/cloud/300081316/microsoft-hires-goog...

[3] https://deis.com/blog/2017/deis-to-join-microsoft/

[blub]

Well put.

Let's take a look at Google: they have the browser (Chrome), they have the OS (Android), they have their cloud, they have a popular programming language (Go), they have the internet gateways (Google search + other services), they have the data (Google analytics, data mining in all of their other services), they have the hardware (Android phones, Chromebooks), they will have the car (Android auto + self-driving efforts), they want to have the ISP (Google fiber), they have the communications (several chat efforts), they have the social network (Google+, several other efforts). I'm probably missing several things.

Now Microsoft: they have the browser (Edge, IE), they have the OS (Windows, Windows phone, Xbox), they have their cloud, they have several popular programming languages (C#, F#, Typescript), they have the internet gateways (MSN, Bing, LinkedIn), they have the data (data mining in Windows and all of their other services), they have the hardware (Surface, Xbox), they have the communications (Skype, Yammer). Once again, I'm probably missing several things.

Apple: browser (Safari), OS (iOS, macOS), cloud, languages (Swift, Objective-C), hardware (iPad, Macbook, iPhone, Apple TV), communications (iMessage, FaceTime). They don't have the data and internet gateways because they're the only ones still holding themselves back from large scale data-mining.

They say the web is open because "it opposes private, exclusive, proprietary Web solutions". But the web is built with Microsoft and Google tools on Microsoft and Google clouds and viewed with Microsoft and Google browsers on Microsoft and Google operating systems. Maybe we can swap one cloud with Amazon, or one device with Apple, but that's the current situation. The ideals of opens-source have been almost circumvented in this age of platforms.

[paulddraper]

Or you can browse a site hosted on Heroku using a Firefox browser on a Linux distro.

The interopretability is huge.

[taocp]

low level programming knowledge is fundamental if you want to work with security research (software vulnerabilities, reverse engineering and exploits) and the job market at the moment is quite big.

[busterarm]

I wish there were a clearer path to transition to that kind of work from web/backend development.

It's something I dabble in and think I'm getting quite good, but unless I start publishing/presenting, I don't obviously see it leading to paid work.

I have a lot of long-time friends in InfoSec but don't really want to exploit those connections to look for work (yet).

[mentalpiracy]

If not for employment opportunities, you should be (gently) exploiting these connections for educational value. They're in the industry and have likely seen examples of colleagues transitioning in from web/backend development. At the very least, they can easily tell you what mistakes to avoid.

I often have the same mindset you do about exploiting connections, and although it's been a struggle for me to change my habits, using people as educational outlets has been something that I've found to be extremely helpful to me personally and also often enhances my relationship with that individual as well.

[Arizhel]

Sure, if you want to work for the government or one of its contractors....

[majke]

I struggle to find good low-level devs. It is surprisingly hard to find good C / kernel / network programmers.

[itsokimbatman]

You're not the only one. My company is desperate for good low level hackers. We can find people who know Python and Java for days, but people with the skill set we need are very hard to find.

[dbancajas]

i am curious how much is your company willing to pay? is your budget around 150-200K or around 80-120K?

[itsokimbatman]

It really depends on where you live. Our headquarters is in a low cost of living area, so 80-120 would be a reasonable guesstimate.

Folks who work in the DC Metro area probably make a good bit more than that.

I'm not actually involved in hiring, just recruiting and trying to find good engineer candidates.

[znpy]

asking the real questions

[tropo]

Well, sort of. It's forgetting taxes, house prices, and more.

Places where you can buy decent stand-alone homes for $50,000 and/or have no state taxes are rather different from Mountain View and Manhatten.

It's funny how people in the pricey places look at a $100,000 salary and not see how it is, all by itself, plenty to support a large family in a home with a big yard. People living that life look back at the city folk and can't imagine how they could ever afford the same thing in a city -- it'd be roughly $2 million (a factor of 40) in San Francisco.

[eli_gottlieb]

Where are you? Want to talk?

[itsokimbatman]

Maybe! We are located near Orlando, FL with offices all over the country. If you are comfortable with low level hacking, assembly, C, etc. then it might be a good fit.

The only hard requirement is you must be a US citizen.

[eli_gottlieb]

Well, I was in fact born in America, and have been comfortable with low-level hacking for years.

[itsokimbatman]

Cool! I saw your LinkedIn profile and passed it onto our recruiter type person. She's going to reach out to you.

[lj3]

Just out of curiosity, what would it take for you to conclude you had found a good c/kernel/network programmer?

[majke]

Assuming a candidate without super-strong experience:

1) Good code sample showing non-trivial piece of code, ideally in C. (craftsmanship)

2) Reasonable github profile, showing ability to contribute to some opensource projects (which means: basic git skills, communication skills, testing) (craftsmanship)

3) Reasonable experience from CV (assuming we're not talking about hiring for junior role) (craftsmanship)

4) Genuine interest and experience in one of crypto or security or networking. (depth)

5) Debugging skills. Systemtap? Valgrind? (depth)

6) Basic understanding of different programming paradigms. Haskell, erlang, scala or prolog count. (breadth)

[AnimalMuppet]

And, just out of curiosity, where are you?

[majke]

The company I work for is in London / San Fran / Austin and couple other smaller locations.

[eli_gottlieb]

Where are you searching?

[majke]

This goes both ways - where do C people look for job. As you know most competent people have one job or another, so it's equally important to be recognizable for the employed bunch. I do a fair bit of blog posts / outreach to good looking github profiles / conferences (smaller and larger).

[dllthomas]

> where do C people look for job

Occasionally, threads about low-level work on HN - you should link to a job post :)

[majke]

Since you insist, this is one of many open positions: https://boards.greenhouse.io/cloudflare/jobs/589571

[dbancajas]

the job market for burger flipping is high. LOL

in a capitalist society, small/rare usually means high-paying! don't make it your career but it pays to know these things when writing your high-level code.

[sbuttgereit]

Small/rare in the supply side is only high paying when the demand is not as small or as rare. It really depends on that relation. Based on the comments I'm seeing... sounds like everyone is talking more about the demand side than the supply side... which sounds either on the edge of oversupply or just plain oversupplied.

Otherwise, I agree with you. Those that understand low level issues tend to, in my experience, just plain understand computing better... even if they're working in the deeply abstracted high level code.

[chimtim]

The demand is quite low. I think if you want a good low-level code implementation, you can easily find one on github. For example, you can easily find a good compression algorithm for your CPU budget or other application needs on github. So you can get away with mostly writing high level code and using these external libraries. Second, the demand for writing really optimized code is not as high as 10-20 years ago. Now, writing reasonably fast, readable code is OK and no one expects you to run gcc -S and count cycles unless you are working in some high frequency trading firm.