Positive correlation between Competitive Programmer’s Handbook and software engineer interviews? Yes.
Positive correlation between being a strong competitive programmer and a strong software engineer? Doubtful.
I have had a lot of experience with competitive programming (competed in NWERC and NCPC multiple times) and this idea that competitive programming makes you worse at software engineering comes up every now and then.
From what I've noticed your statement holds a bit of truth, but I just wanted to add from my own experiences.
To me it seems like those that have done a little bit of competitive programming are good coders. They can code a FizzBuzz in 2 minutes without blinking, they are people that will reach the magic 10K hours of coding quicker than most. However, those that excel at competitive programming can code FizzBuzz in 30 seconds but you will have a hard time understanding what they have actually written. I think that there is a point in competitive programming you pass where you actively have to sacrifice the understandability of your code (and your algorithm library) for increased implementation speed. I also think that in order to excel at competitive programming you have to devote a lot of time to learning algorithmic theory and training on problems which can be solved in <1K lines of code. (Most solutions I've written are less than 300 lines)
The hypothesis I have is that in order to be a great competitive programmer you will have to routinely write bad code and put a lot of effort into solving problems which you will later throw away the solution to and never look at again. Two properties that do not help with software engineering skills. However, if you are only moderately devoted to competitive programming, then you can earn the practical experience (hours of coding) without reducing the quality of your code.
write bad code [...] which
you will later throw away
Implementing a quick & dirty prototype, throwing it away and reimplementing it from scratch is not such a bad software engineering techique: first time to understand the problem, second time to make the implementation nice.
A lot of bad software comes from not throwing the prototype away, but insteat trying to refine it into a product. Result = spaghetti.
I agree. I got into the practice after I picked up F#. I used to create simple solutions in the interactive REPL as mainly proof of concepts. Once I had the basics down, I sat down and wrote a list of possible edge cases, expected user inputs, the format of output and the command line interface for the tool. Then I started coding from scratch and had a good and clean working solution ready. Helped me a lot.
I think part of the misconception comes from the fact that programs written in a competitive programming setting look very cryptic. Variables are not given meaningful names, there're very little comments, sometimes clever tricks are used etc. But people don't realize that in a professional software, they can take the time to write readable code and still make use of their competitive programming skills.
This has been circulated around HN and Reddit several times, and it's disappointing that someone of Norvig's stature would present the data in such a misleading way.
Here's a good explanation posted by "tedsanders" the last time this came up on HN:
"""
All of these claims from Google that say competition
performance hurts or that GPA doesn't matter are
missing one huge thing: selection bias.
Google only sees the performance of the employees that
it hires, not the performance of the employees that it doesn't hire. Because of this, the data they analyze is statistically biased: all data is conditioned on being employed by Google. So when Google says things like "GPA is not correlated with job performance" what you should hear is "Given that you were hired by Google, GPA is not correlated with job performance."
In general, when you have some thresholding selection, it will cause artificial negative correlations to show up. Here's a very simple example that I hope illustrates the point: Imagine a world where high school students take only two classes, English and Math, and they receive one of two grades, A or B. Now imagine a college that admits students with at least one A (AB, BA, or AA) and that rejects everyone without an A (BB). Now imagine that there is absolutely zero correlation between Math and English - performance on one is totally independent of the other. However, when the college looks at their data, they will nonetheless see a stark anticorrelation between Math and English grades (because everyone who has a B in one subject always has an A in the other subject, simply because all the BBs are missing from their dataset).
When Google says that programming competitions are negatively correlated with performance and GPA is uncorrelated with performance, what that likely means is that Google's hiring overvalues programming competitions and fairly values GPA.
"""
I've also heard people involved in Google's Code Jam competition say that Norvig's study was done a long time ago, and no longer really applies.
I think what you said is true. But the main point implied here but I didn't mentioned is the mindset or competence is quite different between programming competition & real work. After all, being good on the job depends more on reflection, going slowly, making things right. ;-)
This is relative to other people who had been hired at Google - there's probably still a positive correlation between those two variables amongst the general population.
Not doubtful at all. You can expect a positive correlation between being an good driver and a good software engineer. Definitely some random person with the CS fundamentals, coding fluency, and creative problem solving ability to be good at competitive programming is more likely to be good at software engineering than some random developer. You're basically saying, high intelligence isn't correlated with being a strong software engineer. Or maybe you think people doing competitive programming are damaged. This is nuts.
Yup. More mental "shortcuts" that seek magic "signals" to avoid both evidence and pragmatic interviewing effort testing both business problem-solving critical thinking and cultural fit. There are no shortcuts. Furthermore, "competitive" programming has little to do with the real world, and it's more about showing off and ego inflation.
Reading between the lines: Problems that are typical of software engineering interviews and in competitive programming are poor evaluation of one's capabilities as a professional software engineer.
I'm pretty sure there is a positive correlation, just because people who tend to be good at competitions usually do them because they like computer science in general, so they learn other stuff too (and they are likely smart). And this is also a reason why this side is tested so much in interviews - it is easy to test and the result is somewhat significant.
Now, most good software engineers would probably not be very good at competitions, because competitions require certain specific skills that are not useful outside of competitions. And that's why good interviews don't put too much weight on algorithmic skills.
It sounds like it may have been awhile since you've interviewed. The modern "good" interview is all data structure and algorithm white boarding nonsense. For me, who is not a competitive programmer, the standard interview prep is spending a couple of hours on Hacker Rank every day and making sure I can delete a node in a binary tree on a white board without so much a missing semicolon. For some reason that signals "good developer" more than a catalog of work on github, big name companies, or experience in general.
I'd rather work with the ones who actually care about the people they're solving problems for vs the ones who can write bubble sort with no reference material, which happens in the real world precisely never. Employers pay developers to solve problems, not necessarily to write the most efficient code possible. I get that it's a craft and it's good to understand what happens under the hood, but you can get pretty damn far solving actual business problems for people without having to care which brand of sort ES2016 uses. These skills in micro-optimizations / textbook CS don't help much with understanding customers and solving their problems at the end of the day, so they're really not that valuable for many business scenarios. I think that a lot of engineers in this industry forget that they're not generally paid to write the most efficient data structures and algorithms for 40 hours a week, and that ironically, attempting to do so would very likely not be the most efficient means of increasing shareholder value. Can't see the forest for the B-trees, if you will.
I don't think knowing details of algorithms is why strong CS skills are important.
Understanding CS teaches a lot about seeing abstractions and choosing between different levels of abstraction and problem solving. It greatly helps modelling business problems as software. I don't think it is important as an end in itself.
I interviewed this week. While there were a few whiteboard coding questions that I personally wouldn't ask, most of the interviews were quite reasonable (mostly algorithmic still, but it was in robotics, so understandable). But yeah, I actually don't understand why people insist on using a whiteboard so much. Everybody has a computer these days and typing is much faster.
At my previous company (not robotics) I conducted a lot of interviews and I think we hit a good balance between algorithmic problems and design/architecture ones. Then again, even algorithms questions can be formulated as real world problems - it help weeding out obvious brain teasers. We also didn't asks people to write code on the whiteboard - there was a separate coding exercise.
I don't even think that asking about algorithms is such a bad thing if you do it correctly and it's not all what you do. But perhaps too many people just throw a problem at the interviewee without and help or guidance.
Overall, conducting a good interview is hard and it is not a primary job of most engineers who do it, so it's not surprising there are quite a few bad interviewers out there.
I observed a correlation between competitive programmers and those who come across as just hacking at it until it works in interviews. If your interview system is looking for people who think about things then it probably already biases against many competitive programmers correctly.
Your second assumption is only valid for people who conflate the two disciplines. If you don't recognize that competitive programming and software engineers have completely different goals and requirements, then being trained in competitive programming may very well lead you to bad programming practices - e.g. single-letter names are usually fine for competitions, while at the same time usually horrible for engineering.
However, if you recognize the differences and use the skills you learn in the two as complementary, you can be better at both.
Maybe not, but just skimming through the topics looks like there is a lot of overlap between competitive programming and popular job interview questions.
The risk is that you might be less useful than someone who is particularly adept at system design.
It seems most problems are actually not sorting, searching, or finding the optimal whatever. Maybe it's just the bubble I work in, but from my perspective it seems that most programmers aren't addressing a problem of the form, "the obvious solution to this well-defined problem is too slow, please have a clever insight that leads to a faster one."
The problems we work on are instead of the form, "please model this sprawling and subtle domain with reasonable fidelity and in a way that'll handle future changes to the domain."
"Please satisfy these five dozen individually trivial requirements in a way that gets every corner-case interaction exactly right, and won't turn into a nightmare when there are a dozen more next quarter."
"Please decompose this problem in such a way that 10 different people can work on it in separate parts of the codebase in parallel."
"Please take this problem that's solved for one machine and make it work over an arbitrary number of machines, and make it reliable under all the weird and abusive scenarios that a few years of usage in production can manage to throw at you."
"Please design a monitoring and dashboarding strategy that will identify all outages immediately while not overwhelming the oncall with false alarms, and provide first-class instrumentation, debugging, and remediation tools so that someone new to the codebase can find out exactly what went wrong and fix it in the middle of the night.
It's not at all uncommon to deliberately ignore the optimal algorithm in favor of the readable algorithm. We usually try to keep cleverness behind the curtain of abstraction (RDBMS, standard library, etc). Of course, someone has to build them, but then they are widely reusable.
Of course, people who can do all of the above while interacting fluently with the code for the optimal algorithm are so incredibly highly paid in the Bay Area that they can buy houses.
Think of it this way: there's only a certain amount of productive hours available to people for side programming, and either you spend a majority contributing to open source/building prototypes/learning new technologies/getting your hands dirty; or you spend a majority of the time solving artificial problems to get a leg up on programming interviews. It's a catch-22 if ever there was one.
No, but by definition, any non-trivial software product (or academic CS paper, for that matter) is the outcome of a collaborative process, not an artificially time-constrained hack-a-thon/competition. You have to put things like this in their proper place, and let them be what they are. (I downloaded the pdf, btw).