People with Engineering / Comp. Sci. Degrees are not in short supply. Exceptional engineers and programmers are. Its almost like the difference between kids who want to play football and the people with talent to make it.
While this is true, I have found that a developer who can do the following to be very rare:
1. Take a set of requirements, and first identify those reqs for any underspecified elements, or elements where the underlying tech would limit the functionality.
2. Implement the reqs according to the now agreed upon spec
3. Provide adequate documentation and tests for the new implementation.
4. Perhaps most importantly, and rarely, implement the reqs in a manner where the code is not brittle (some small req change doesn't require a huge overhaul) and where other developers can quickly get into and understand the code.
That is, even if you don't need a rock star programmer, if your code base is going to live on for more than the life of the person originally writing it you are going to find that people who can fulfill this role are quite rare.
All of these things could be easily overcome with better training. The way I see it, the software industry in the US invests close to nothing in training and expects to receive "talented" developers by just selecting a small proportion from a large pool of candidates. This is a lazy and irresponsible way to create a workforce. The time and effort they spend in interviewing could be very well spent on teaching best practices to new employees, so they would be much more productive, instead of just expecting that they somehow "grok" these concepts from the get go.
I think the implication is that you're supposed to learn all of that on your own. That's at the very minimum-you're supposed to eat, breathe, and shit hacker culture and programming in your spare time.
> All of these things could be easily overcome with better training.
I think 1, 2, and 4, requires lots of real world experience in addition to training.
> The time and effort they spend in interviewing could be very well spent on teaching best practices to new employees
Assuming that just teaching best practices is all that is required, which I strongly disagree with, a lot of companies (especially startups) don't have the time or the resources to burn on extensive training. The marketplace is fiercely competitive, and they typically need people on the ground right away.
> the software industry in the US invests closing to nothing in training
Many large tech organizations (Google, Microsoft, etc.) invest a huge amount in training. The time it takes for a new hire at Google to get productive is usually about 6 months to a year. All still, they have a high hiring bar that focuses on very strong fundamentals. (They also work very closely with universities and colleges on direction, curriculum, and funding.)
"The marketplace is fiercely competitive, and they typically need people on the ground right away."
" The time it takes for a new hire at Google to get productive is usually about 6 months to a year."
It cannot be both ways as you please. Either these companies are investing in training or not. If they are, then things like requirements gathering, creating specs, and providing tests should not be something difficult to do. This is a very important part of the job, but is not a part that requires super-intelligence.
Everything points to the fact that these companies do not invest in training and are just trying to select a small number of people out of a large pool.
I don't think this is difficult, but I agree that it is rare. There is very little pressure encouraging people to become one of these programmers. I'm one of them, but it's difficult to actually leverage this into a higher salary unless you got involved with a larger company straight out of college. The interview process doesn't expose this, and unless you write a hit open-source application there isn't any way to prove it.
Instead, employers - even technical people who interview you - play buzzword bingo, indicating that you would have been better off spewing new frameworks into your codebase unnecessarily. The personal incentives for a programmer are completely misaligned with the actual interests of the business.
The approach that would incentivize this behavior would involve giving raises to people who write the sort of code you want to see, but instead raises often come from dazzling strangers who don't know the long-term consequences of your code (ie, getting a higher offer and asking for a raise).
There doesn't seem to be a way for a company to evaluate the "goodness" of your code in a meaningful way. And even if they did, you will NOT get a raise without asking for it. OTOH, there are those, like you mention, who will leverage other job offers to get a raise and such...so the only solution seems to be for those who write the good code to engage in similar tactics, regardless of the disdain they might have for them.
4 4 and 4. With no requirements, if you have an implementation of what is actually needed and it is not brittle you are all good. People who actually know what they are doing and don't write brittle code are more valuable than rock stars. I have seen too much code by rock stars based on the latest coolest frameworks and it has all been thrown out. Same for people who don't actually have enough skill and 'just' get the job done. One change and it all dies, turns to a pile of steaming poo or gets thrown out, only to be developed again using this week's cool new js framework. Less brittle code is smaller, simpler and involves as much deleting as adding.
Thank you for posting this, and am reading it very closely. However, I can't quite follow what you're saying with the first point, which is probably my fault. Would you mind putting it in other words?
I agree with the literal sentence. I disagree that with its implied extreme. Most jobs don't require rock star programmers. Most jobs require people that can do more than cobble together a series of StackOverflow answers. This implies that most jobs need to have competent developers.
From my limited experience, competent is getting harder to find. I go into a JavaScript SPA almost daily. I'm not a big JavaScript guy, but I grok functional programming and the language. I constantly see things that make me go, "Huh?" This isn't that they're doing advanced JavaScript things. It's that they are doing it wrong (one person used a regular for loop to side-effect create a new array). These poor practices add up in general, and specifically in an AngularJS app where you've got to accept a layer of magic in the view binding.
Hopefully we get better at competent, especially in dynamic, mostly typeless languages such as JavaScript.
I really can't agree or disagree with what you're saying, you are talking about very specific things that I'm not sure say anything more than your biases and experiences.
Maybe they don't, but maybe they do. How many of those jobs wouldn't exist with access to a more talented team? As an analogy, I don't require a phone to communicate with my mother, but if I didn't have the faster method, I'd have to travel to see her.
That's not really true - with more and more tools and development philosophies coming available, we need more and more experienced programmers with the judgment to choose between them and apply them to the problem at hand.
That's not actually how high-functioning teams work. If you have one person make all the decisions and pick all the tools, you will have one highly-productive programmer and 9 disengaged programmers who go through the motions of writing code but usually end up creating more work that the one highly-productive programmer will have to undo. End result, he gets burned out, the product never ships, and the 9 disengaged programmers never actually learn how to program.
I've been on teams where everybody understands how a compiler is built & when you'd want to write one, why & when performance matters, how Lisp idioms can translate to C++, when you might want to machine-learn a model and why you might want to do it manually first, and many other basic CS concepts. The difference is night & day from teams that are filled with people who just follow a tutorial and copy/paste examples from StackOverflow. In general, I've found that a team moves at the pace of the slowest member of the team - fill a team with 1 smart guy and 9 folks who just follow their leads and you might as well have just hired the 1 smart guy.
You should understand that your logic can be used to prove that any profession is in short supply. Exceptional accountants are in short supply, so we should hire them in droves from India. Exceptional cab drivers are in short supply too, so we should have a visa waiver to bring them from Europe...
>Exceptional people in any profession are in short supply. Isn't that a true statement?
It equivocates on whether by "exceptional" and "short" we mean absolute standards or relative ones. If we use relative terms, "exceptional" (say, the top 1%) people are by definition in short (only 1% of the total) supply.
Of course, 1% of a large population can still be quite substantial.
Your statement is ridiculous. I'm not going to discuss the demand for accountants and cab drivers or the differences between engineering job and cab driver, but your assumption of there are plenty of exceptional accountants in India and exceptional cab drivers in Europe is unfounded.
Why would Indian accountants be experts with U.S. accounting practices and tax laws? Would they be amazing at dealing with customers and auditors? What makes you say they'd be more cost effective than American accountants?
Why would European cab drivers be more effective than American cab drivers? If you put one of those crazy taxi drivers from Eastern Europe here he'd get arrested within hours. Would they provide better customer service when they don't even speak English?
Where as for Engineers, good engineers from China and India are just as good at their subject as anyone and culture barrier is not really a big concern when you are dealing with pure technology, so of course it makes sense to hire top engineers from other countries.
If good engineers from China and India (or really anywhere) are just as good at their subject as anyone else then preference should be given to the citizen, but it's actually the reverse in practice, and that is by design.
If the foreign engineers are really that good then why do most of them do low level work with low status positions for 70k/year? Sounds like poor use of the "best and brightest" to me...
There is no preference when it comes to hiring Top Tier engineers. Here at Google we try to hire everyone that can pass the interviews, if you are a U.S. Citizen that doesn't require Visa sponsorship then it's all the much better. I've never ever heard of anyone losing their job to foreigners in top Silicon Valley companies.
I never said foreign engineers are better, I said top ones are just as good as top ones here. Most of them work in low level positions for 70k a year but that's true for most American engineers as well. Cost would obviously play a big role for jobs of those level.
But I was replying to the comment about exceptional talent, not the overall job market.
What makes you think they WOULDN'T be as good? I've casually worked with some top tier Engineers from Baidu and Alibaba before this job and I'd say they'd have no problem landing a good position at Google or Facebook any day.
When you start looking a the implications of the fallacies of "talent in short supply", all these crazy ideas start to crumble. A large number H1B visas in this country are used by temporary agencies that bring workers to do jobs for low pay. This doesn't seem to be the pool of bright programmers that their proponents have in mind.
A lot of foreign workers are willing to work for less money, in exchange for a H1-B visa.
Why are they willing to do that? Because H1-Bs are scarce.
Why is it scarce? Because a lot of Americans have decided that H1-Bs should be limited, because they fear foreign workers coming over and working for less money.
There is nothing ridiculous in this statement. You are talking about "exceptional" workers, not about normal or even bad workers. Exceptional workers are in short supply in any profession. According to the logic of software industry, the solution should be to import workers from other nations, because then we would have access to the best of the best.
Of course it is not like that. Software engineers from India or China are on average as good as the ones in the US. If something could make a difference is education. Now, if engineers in the US are not as good the ones coming from Asia, the country should do something to fix this issues first by paying more and educating them in a superior way.