[shantanubala]

I think programmers tend to underestimate the difficulties involved with becoming a good programmer because once you're good, you only see the even steeper learning curve ahead of you.

Some of the smartest people I know work in other domains: biology, chemistry, and even physics. They are sometimes baffled by tasks that seem trivial to me, and I'm under no impression that I'm more intelligent than them. I simply specialized and focused only on programming, while they program to accomplish other tasks in their domain of expertise.

Can this last forever? Of course not, nothing lasts forever. But wondering why the wealthiest corporations in the world pay their workers high salaries is perhaps like wondering why water is wet. Software has a low marginal cost, and the rest is basic incentives for the corporations.

[TeMPOraL]

> I think programmers tend to underestimate the difficulties involved with becoming a good programmer because once you're good, you only see the even steeper learning curve ahead of you.

Good programmers I know also overestimate the skill needed to earn high salary in this job. You don't have to go up the learning curve much; these days, you just learn yourself JS a little bit and go for a webdev job, making shit code and still earning more than most people in a given country.

> But wondering why the wealthiest corporations in the world pay their workers high salaries is perhaps like wondering why water is wet. Software has a low marginal cost, and the rest is basic incentives for the corporations.

Nah, that's like wondering why is this ice block sitting on a hot plate and still solid. The answer is: because it just got put there, and it'll melt in a moment. So too, will end high salaries, as most low-hanging fruits get eaten by software, made by mass-produced cohort of programmers.

[spamizbad]

> Nah, that's like wondering why is this ice block sitting on a hot plate and still solid. The answer is: because it just got put there, and it'll melt in a moment. So too, will end high salaries, as most low-hanging fruits get eaten by software, made by mass-produced cohort of programmers.

Our industry has its share of cycles, but this, in my view, is largely wishful thinking on the part of people. Nothing wrong with optimism but...

Every 5-10 years there's a "technical shift" that forces everyone to reevaluate how they build software or more importantly what they build, and the race starts all over again. The ice block is removed from the hot plate is replaced by a bigger, colder block of ice. And when these technical shifts aren't taking place, the bar for what constitutes as "good software" inches upward.

If your standards for acceptable software were frozen in time in 1985: using modern hardware and software toolchains, you could accomplish in one day what used to take a small team an entire month. But if I delivered, say, what passed for a "good enough" database in 1985, it would resemble someone's 200-level CS final project rather than a commercially-viable piece of software.

[kerng]

I have not noticed these technical shifts per se. What I have noticed is that mature engineers move on and do other things and new ones reinvent the wheel with some new fancy language or term which then becomes the new way of doing things, and the cycle repeats. Sometimes there is a great deal of value when a new level of abstraction happens but I wouldnt call this a shift, it's just progression.

Many of the underlying problems and solutions exist for decades. Database systems you mention are a good example of this.

[spamizbad]

When I talk about shifts, I'm referring to things like the proliferation of smartphones and tablets, which increase the net-demand for software and along with an entirely new specializations of knowledge.

While there are some key concepts to things like databases, the fact remains that your 1985 database would not be considered sufficient in todays world: It would have too many limitations, lack features we now take for granted, would not scale to modern data requirements, etc. Supporting all that "modern" functionality is non-trivial and requires a huge amount of effort. You can't just say "Well, we figured out space and computationally-efficient hashing, so relational databases are well on their way to being feature-complete"

There's a reason we haven't stuck with 1.0 on our platforms, and it's not just security or a desire for a bigger version number: New demands required new functionality and new ways of building things.

[jimbokun]

"When I talk about shifts, I'm referring to things like the proliferation of smartphones and tablets, which increase the net-demand for software and along with an entirely new specializations of knowledge."

iOS was basically AppKit, so anyone already developing for the Mac knew most of what they needed to know to develop for iPhone.

Pretty much every programming innovation is incremental, and doesn't require throwing out all of your previous knowledge and starting over.

[JKCalhoun]

> iOS was basically AppKit, so anyone already developing for the Mac knew most of what they needed to know to develop for iPhone.

Maybe. But AppKit was not the Mac Toolbox.

When my career began being good at memory management was a skill to be proud of. I would say now, being good at concurrency is a skill to be proud of.

I don't really have to worry about memory management any longer but didn't worry about threading when I started my career.

As I see the younger generation entering the programming field I wonder in what ways the craft will be different when they've had a few decades under their belt.

Will parameter tuning datasets for machine learning be the coveted skill? Who knows.

[pennaMan]

>so anyone already developing for the Mac knew most of what they needed to know to develop for iPhone

But the demand for developing in AppKit suddenly increased by orders of magnitude.

[convolvatron]

i'll just leave this here https://books.google.com/books/about/Distributed_Databases.h...

[nostrademons]

MongoDB didn't become a public company through innovations in fundamental distributed database technology or even through good engineering. They became a public company because once Javascript became adequate for building client software, there was a strong incentive to build MVPs using the same data structures from client to server to DB, and once you build an MVP that gets adoption there's a strong incentive not to switch databases.

That's the sort of shift in the environment that the grandparent is talking about. Fundamental CS tech was arguably better in the 1970s and 1980s, because it moved more slowly and you had time to get the details right. That doesn't matter if you're building say a mobile Ethereum wallet in 2018, because you're building for the user expectations of today, they don't care about data integrity or security as long as it doesn't fail during the period where they're deciding which tech to use, and software that solves the problem (poorly) now is better than software that doesn't exist.

[joshuamorton]

This predates the formulation of the cap theorem by ~15 years, so I doubt it contains everything relevant to a modern distributed database.

[kerng]

I'm not sure why you get downvoted, I'll upvote you.

The "modern" database systems are now going back to the exact design principles that the books you refer to solved long time ago. There is tons of research, dissertations,.. that focuses on this from decades ago.

Its just now that the new systems realize that these problems actually exist.

If you dont know the history of a certain field and what came out, you repeat and make the same mistakes again. This seems to also apply to software engineering.

[lumost]

In maturing parts of the software industry you'll often see a desire to stay with the times in order to maintain a competitive edge, re-inventing the wheel often looks like full/partial re-writes of a system for minor marginal gains.

A great example of this is the evolution of FB/Google/Amazon. Portions of their core tech have been completely re-written over the years for marginal gain, but there is a large premium to being the best in tech.

In other parts of the industry every new cycle enables some new area of tech, and those marginal gains become the minimum bar for entry. e.g. Deep Learning and Computer Vision, distributed systems and cloud computing/SaaS.

[hopler]

You overestimate the quality and reliability and expandability of those old systems. The 1998 10 Blue Links tech couldn't support the functionality and scale of Google today.

[gaius]

your standards for acceptable software were frozen in time in 1985

Haha in 1985 Amiga had a multitasking GUI desktop in 512k and 7Mhz. Now we have millions of times more computational power and struggle to deliver an experience as crisp and satisfying.

I wish people still made software like it was 1985, that actually used the full power of the hardware for something useful...

[spamizbad]

Yeah but Exec/Kickstart/Workbench was missing some basic niceties that are table stakes today:

- No process or kernel security (processes could rewrite kernel code)

- Processes could effectively disable the scheduler

- Supported only a single, fixed address space: both a massive limitation and a performance hack that made the system bearable to use

- Single-user

- No security model

There are embedded applications these days where not having these features are deak-breakers. Let me assure you: if you re-implemented the original Amiga OS feature set it too will be screaming fast. The tricky part is keeping it fast once you start adding protection and additional functionality on top of it.

And largely what happened when you tried to implement more complicated applications on top of these primitive systems is that they would crash the entire system, constantly.

[makapuf]

That and the fact that amiga was clean room design. IBM Pc was already old and even x86-64 won over itanium. Backwards compatibility has a cost but also gains. Amiga wasn't even compatible with c64.

[gaius]

That and the fact that amiga was clean room design. IBM Pc was already

Well the PC wasn’t backward compatible with CP/M so that’s an odd critique to level at the Amiga.

[pjmlp]

Itaninum only lost because Intel did the mistake to allow AMD to design x86 chips.

[gaius]

more complicated applications on top of these primitive systems is that they would crash the entire system, constantly

That’s chicken-and-egg. Why do modern apps with actually quite simple functionality need all this vast power, the GHz and Gb? Because it’s there. Why does software crash? Because the OS let’s it with nothing more than inconvenience to the user.

Amigas were actually quite usable, they were stable enough for complex software to be developed and real work done. Same for ST’s.

[gcb0]

your whole point is extremely backwards! software in 1985 was more "complete" than today. Even CD roms shipped with magazines had to support more windows variations than "good software" today supports of browsers! not to mention that every corporation with a software team also had usability and QA teams. Software quality and resiliency was much better than today. Then in the 90s it took a dive because online made "first to market wins", and it have been downhill from there.

so, your software concepts from 85 would be overkill today, not lacking.

[antt]

In 1985 you target one machine with one thread and one level of memory.

Today I'm working on systems with memory latencies from l0 cache all the way to tape storage. And it's getting worse.

[pjmlp]

Not when coding machines like the Amiga.

[zrobotics]

Agreed, and the games industry may be the clearest example of this. In 1985 the developers had to actually ship a finished game, there wasn't the opportunity to release a half-finished product and update it later. Compare to Fallout 76, which was very clearly unfinished at launch.

Edit:typo

[hopler]

Games in 1985 shipped with bugs baked in that people were just stuck with. Civilization had a broken Gandhi, and still shipped 4 different bugfix versions.

[zrobotics]

No game (or any large software project) is ever bug-free, but the standards were higher than they are now. Fallout 76 was literally unplayable on launch-a fairly early mainline quest was broken, so it wasn't possible to reach the endgame. Was civ unbeatable on release? Sure, ghandi was bugged, but that was entertaining enough they kept the bug in the sequels.

[dmd]

Neither Civilization nor Windows existed in 1985.

[hnzix]

The anecdote is they deliberately left in nuclear Gandhi for the lulz.

[packet_nerd]

> inches upward

I wish it inched upward... So much software now-days is bloated crap.

[ralphc]

Preach, brother/sister! My career started in the early '80s and I say this all the time in comments here on HN. You had less to work with but less was expected of you.

[sethammons]

I think you analysis is fair, but Glassdoor data is a bit off. At my work, I filled out salary data on Glassdoor and it never showed up. Maybe they thought it too high?

[zanny]

End user tasks might be able to be automated into dumb programming jobs, but companies have tried for decades to down-source their programming and it always falls flat on its face.

Even if you have library support to hand hold your budget coders, even if you use a lot of them, even if you give them all the time in the world, they will produce more complicated, less coherent, less stable, buggier, and harder to modify, improve, or iterate results than better coders who understand the problem better.

That means that no matter how little you pay up front you end up paying more in the long run throwing more man hours and money at fixing the mess made. A good mess is easier to maintain and improve and costs less over time. A mediocre / bad mess takes substantial efforts to maintain and iterate on.

Its also probably a domain impossible problem to remove the ability for any coder to make bad code. If for no other reason that in any programming environment you can never stop someone from iterating by index over a binary searchable tree for a specific element in it, or you can't stop someone from turning a float into a string to truncate the decimal part to then reinterpret the result as an int. But if you don't give them the tools - the integer types, the data structures, access to the bytes in some form or another - you aren't really programming. Someone else did the programming and you are just trying to compose the result. A lot of businesses, like I said, can be sated on that, but its still not programming unless you are in a Turing complete environment, and anyone in such an environment can footgun themselves.

[therealdrag0]

> and it always falls flat on its face. How so? What about outsourcing in general? Where I work wanted to save money, and now we have 4x as many engineers in India. Most of them suck, and we have to shepherd and correct their mistakes, but business is viewing it as a success. They are creating features at a higher rate than we were before. Every other large company also has teams in 2nd world countries because they're cheap. At some point this labor will be both cheaper and qualitatively comparable to US talent.

[Animats]

"any programming environment you can never stop someone from iterating by index over a binary searchable tree for a specific element in it."

SQL solved that problem a long time ago. You write SELECT, the query optimizer figures out how, using any available indices.

[gldalmaso]

I think that misses the point. SQL does not prevent a junior dev from selecting many more rows than he needs to and then proceed to iterate over them naively.

[xiaodai]

just use Rust

[zanny]

I know its a troll comment but it really illustrates my point. Rust feels great when you know all the paradigms and complexity involved in writing Rust code because for a seasoned veteran it does everything sensible and right. But for your average person trying to get into Rust is a nightmare of fighting the borrow checker on all your misintended mistakes that would at best be silent undefined behavior in other languages. Top it all off with how expressive Rust is relative to most C like languages (remember, we had an entire generation who thought function objects were alien because they were taught Java) and its fantastic if you know what you are doing and a totally lost cause when you don't.

[tenaciousDaniel]

Ironically, knowing that I can get a good salary with relatively shit skills makes me want to up my game. Because it seems like a situation that is inherently unstable.

Eventually it has to change (imo), either through companies becoming more scrupulous in their hiring, or through a massive flood of new devs.

[danielcampos93]

If you take a look at most schools CS is now one of the most popular majors. When I graduated ~3 years ago the class was ~100 per grade. I went back to recruit and class size was ~600 per grade. Talking to profs I know all over the country this seems to be thematic. The supply curve is about to shift.

[dropit_sphere]

I think it will be a massive flood of new devs (maybe we're in it now?) -> overvalued tech stocks -> crash -> stricter hiring.

It's not that tech, even done by shitty devs, isn't valuable. It's a question of whether the market can control itself, which I'm pretty sure is no.

[arcturus17]

> stricter hiring

Stricter hiring on the low-end perhaps, because in FAANG and the companies that copy them, hiring could hardly get any stricter...

[xiaodai]

You will get a massive flood of wanna-be new devs. But economic theory would tell us that high salary attracts more talent and even the companies will have more people to choose from they also become more selective so the riff-raffs still won't get a job. So only the good ones will get the plum roles; of coruse a few bad one fall through the cracks, but overall devs are smart people.

[brandall10]

I remember back in the early aughts after the .dom implosion and the resultant uptake of outsourcing. There was a pervasive attitude that the high paying software jobs were not going to come back in any meaningful way. Yeah, well...

There will be a day, but when is hard to say. Thinking it's right around the corner is akin to the belief we're on the cusp of true AI. We're more pessimistic today than we were in the mid 80s. And non-programmers were programming with hypercard, filemaker pro, VBA, etc, back in the 90s.

There are of course former well paying jobs such as old-school front end devs (html/css/sprinkle of js) that are largely commoditized, but that's a given considering the low barrier to entry.

[zjaffee]

Except the high salaries won't end given the level of productivity found within the technology industry. Tech companies as it stands today are some of the most profitable companies out there, it's this profit that enables them to further compete for talent and pay people more.

The same highly productive nature per employee is found in virtually every other high paying industry, most of which have not seen the pay for the higher end of those in the industry fall over time.

[ljm]

> these days, you just learn yourself JS a little bit and go for a webdev job, making shit code and still earning more than most people in a given country.

With these jobs in particular, what I see is that the definition of seniority has shifted to 'knowing the latest tech'.

So a junior dev who's just got to grips with React has become a React Developer, and they are now relatively senior in that field. The experience isn't transferable to other parts of the software stack though, it's too heavily tied up in the browser. So they end up as a super-specialised frontend dev.

It'll pay pretty well until the tech becomes obsolete, unless that kind of person enjoys maintaining legacy code.

[MRD85]

I don't see that mass-produced cohort arriving any time soon. My city has 1 developer for every 3 jobs and it's getting worse.

[TeMPOraL]

Universities are already producing those like there's no tomorrow, and we now have bootcamps that contribute some more. The only reason there isn't a glut of programmers visible is because the industry is growing even faster. But I can't imagine that growth lasting much longer.

[miguelrochefort]

What city is that?

[MRD85]

Sydney. This statistic was given to me this week, I actually figure that it's 1 person for every 3 vacancies but who knows.

[onlyrealcuzzo]

So then when are doctors going to start making less? And investment bankers, too?

[organsnyder]

Doctors are already seeing commoditization as healthcare systems consolidate and adopt standardized workflows (aided by EHR implementations). And nurse practitioners and physicians assistants are doing much of the work once done exclusively by doctors, but with less training and lower pay.

It wouldn't surprise me if index funds have caused a decline in earnings for investment bankers.

[Jorge1o1]

Yeah, but the relatively poor performance of index funds in 2018 (no gainz) will lead to capital flowing back to ibanks and hedfe funds

[stickfigure]

I think the comparison to biology/chemistry/physics is interesting. Perhaps even more than software, there's a huge spread between the value of low and high performers - the best scientists make new discoveries that can be worth billions.

On the other hand, if you think the software industry has a hard time figuring out (at hiring time) who the high performers are... science is driven by serendipity. Nobody can predict who will find the billion dollar discovery. Not even past performance is a reliable indicator.

So it makes sense to me that the salary spread in science is relatively even. If they could reliably figure out who to dump money on, they would. On the other hand, the FAANG companies clearly believe their hiring practices can select out the high performers... and perhaps they are right? If they're paying 3-4X what everyone else does, they expect to get at least 3-4X the value.

[titanomachy]

I've worked with good people at every company I've been at... but the nice thing about being at a top company is I never have to deal with totally incompetent or helpless people. Nothing frustrates me more than having my job responsibilities include training someone with no initiative.

The selection process seems to do a good job of keeping out the lowest tier at least, although we openly acknowledge that we miss a lot of good people as well.

[mgkimsal]

> Nothing frustrates me more than having my job responsibilities include training someone with no initiative.

Years ago, I worked someplace where a colleague was tasked with working with another developer on project X. After about 15 minutes it was clear the other developer ... wasn't? A web project, and this person had been employed as a "web developer" for at least several months. Questions like "how does this information in this browser get back to the server?" came up.

Colleague goes to manager and says "I can hit the project deadline, or I can make sure other_dev learns the basics enough to be able to contribute and understand projectX, but I can't do both by the deadline. Can we move the deadline back a few weeks?"

No, and no. Train other_dev and hit deadline.

Deadline was hit, other_dev moved to another project afterwards, and was pretty much as ineffective as before, but colleague was then saddled with this reputation of being a 'bad mentor' because the next team learned other_dev didn't know how things worked. Why the hiring manager wasn't tarnished... who knows?

[hello_moto]

Sadly reputation comes from result.

He was given 2 tasks and he only delivers one result.

I know this sounds... not ideal... but it is what it is.

His manager probably has to operate at the same level of expectation: given 3 tasks by his manager (or director), either you finish all 3 or you're less dependable.

[trinkwallopy]

That just sounds like a place that doesn’t have smart people. He was done a favor because it forced him to move on to somewhere better

[mgkimsal]

Isn't the hiring manager tasked with "hire someone with basic competence"? And they failed? But their reputation/credentials don't get called in to question?

[someone454]

Science doesn’t reinvent itself every couple of years either: new discoveries build upon a foundation of old discoveries. Software is more like the fashion industry.

[sjg007]

I think they expect to not have to face another Google or Facebook to compete with by hiring everyone at 3-4x. Then employees just rotate among the big techs. The big techs then decide where they want to compete with each other... e.g Netflix and amazon video/prime... so if stocks decline I’d expect a rise in bonus or base pay or more stock.

[bdcravens]

Part of the issue is that the metric for "high performers" can be well out of line with skill, at least as measured as a product of profitability.

[ball_of_lint]

One way that they may be getting 3-4x the value is in the long term. Although I've only worked as a programmer, I'd expect that the impact of having one negatively valued programmer is far larger than having one negatively valued chemist or physicist. Legacy code can often be years old and far outlive the jobs of the people who wrote it.

[analog31]

Oddly enough, I think learning to program is easy, but only for a few people. And those are the people who are motivated to learn it as an end unto itself.

I was motivated because my older brother, and my mom, had already learned how to program, and they were quite excited about it. After getting past a few familiar conceptual hurdles, it became very easy for me to learn programming myself.

People who are only motivated by the money, or under pressure from others, have a harder time, because their curiosity and drive aren't activated. There's some sort of valve that lets the knowledge into your brain, that has to be opened.

For the most part, the people I know who seem to be motivated by money itself are not so desirous of getting rich per se (many are already rich), but are actually interested and curious about money in the way that I was curious about programming.

I don't program for a living today, but my ability to program is definitely a force multiplier for my work. It has either improved my earnings, or improved the continuity and longevity of my career.

[abledon]

"""I don't program for a living today, but my ability to program is definitely a force multiplier for my work. It has either improved my earnings, or improved the continuity and longevity of my career."""

may I ask what domain you are working in? Can you give some examples of how you've slipped in some programming knowledge into other job tasks? I love to hear people's anecdotal problem/solution approaches. Was the programming side of it actually slipping in some VBA/chrome extension/javascript or was it more of just an 'analytical' approach taken to a business decision.

[analog31]

My background is in math and physics. While studying those subjects in college, I learned programming on my own. Today, I develop technology for fancy measurement and control equipment. When I say I don't program for a living, I mean that it's not my job title, and my managers may actually be unaware of the role of programming in my work.

I use programming extensively as a problem solving tool, for things like data analysis, modeling, automation of experiments, and prototyping. Almost all modern equipment is electronic and computerized. To be capable of rolling out an MVP on my own, I program.

You will rarely see my computer without a Jupyter notebook on the desktop. ;-)

In addition to working in a computerized field, program code is just a super powerful way to express ideas. And the disciplines of good programming practices (yes, learn them) provide ways to organize the innards of complex things, so they actually have a fighting chance of working and being right. Plus, it's fun.

People who work as full time programmers may make more money than me, but I'm not sure that I can do their jobs. When thinking of any profession, a person should not only look at the cool, fun stuff, or the money, but what the actual daily grind looks like, because that's what you have to survive.

[tanilama]

> I think programmers tend to underestimate the difficulties involved with becoming a good programmer because once you're good, you only see the even steeper learning curve ahead of you.

But that is no definition of bubble. Bubble, at a very basic level, means there is a lot of capital flowing within, it has little to do with whether how difficult your job is.

[Pamar]

This is... debatable. I was around in 1985 and if I had proposed using "an eventually consistent schemaless DB" to tackle a bank or a manufacturing project I would have been laughed out of the room. And not because it sounded too good to be true, mind you...

[woolvalley]

Funny thing is, that is what ACH and many other banking systems are today, or at least back then. "Reconciliation" can take days.

[sedachv]

> I was around in 1985 and if I had proposed using "an eventually consistent schemaless DB" to tackle a bank or a manufacturing project I would have been laughed out of the room.

Medical record systems were running on MUMPS (schemaless) and being eventually consistent (records were keyed in from paper forms) long before 1985.

[ummonk]

Does anyone do that, at least on the software side? Obviously there are consistency issues with e.g. non-instantaneous bank teller actions, but those are human inconsistencies, not software inconsistencies..

[Pamar]

What I mean is that some of the powerful components we can leverage today solve problems that did not exist in 1985, so having these available then would not really help.

I mean: what would you use Node.js for - in 1985 - even assuming you had access to a system to develop and test stuff made with it?

[sedachv]

> I mean: what would you use Node.js for - in 1985

In 1985, your hypothetical bank would be running VMS, which had asynchronous IO system calls as the default. There was no need to "invent" something like Node.js.

[sam0x17]

I agree, but the real bubble that has popped is that non-techies have become more selective in judging the skills of programmers. Gone are the days where being able to stand up an FTP server and display a mysql query result in a web page means you can write your own ticket into any tech job in the world.

[shams93]

Most programmers are not that well compensated. I never made more than 120k a year tops, I'm almost 50 years old. Not everyone is young and pretty and attractive to the Googles. The number of employers who compensate like this are tiny you can count them on one or two hands. 98% of programmers never make more than 100k. With nearly 30 years experience I build things that would bill out to $160,000 a month for an outside team to produce, but I live in an 80 square foot roach infested apartment and can't afford a car.

[dpau]

If your work is that valuable then perhaps you’re selling yourself short? In my 40s now and have definitely encountered age discrimination but found remote work once I updated my skills. I hope you get out of that dump!

[softawre]

Do you live in a really high LOC area? Or are you exaggerating?

[amrrs]

Absolutely right and something we see in Data science too. Coding is a domain where we work cross-architecture. I can't be just a useful Python programmer - I can only be a useful Python programmer in Marketing automation or some Business associated with it.

The fat package makes programmers not realize this.

[natalyarostova]

As a data scientists who considers himself moderately good with the pydata library set, I realized the other day if you took those 5 libraries away from me, I'm not sure I have that much to offer in terms of my programming abilities. I don't necessarily feel bad about that, although it did give me pause.

[simonh]

I’ve known people who built lucrative careers on expertise in specialised technologies, and others who’s entire careers disappeared because they were too specialised and the technology got deprecated. You might be fine, but I’d definitely recommend diversifying your skill set a bit. Being a generalist has served me well.

[SZJX]

I would say every capable programmer should be a "generalist" in a sense and be able to transfer their skills with relatively little effort. If they can't, they are not really that good at the abstract concepts. Still this doesn't prevent one from focusing more and specializing more on one field/language that they love and be as familiar with it as possible.

[autokad]

the libraries have grown nicely over the years, though I am surprised they still lack some basic stuff.

why in 2019 is there no good way to visualize a decision tree? having to install, configure, and get working graphviz feels very hoaky.

I prefer python, but R still does some things really well that the python libraries are just not up to par

1) anything geospatial, including drawing maps. here is a list of projects my students did to give examples: https://pennmusa.github.io/MUSA_801.io/

2) time series

3) linear models, is it so hard to give me a good summary?

If anyone knows of any packages that do these >= R, I'd love to see them :)

[stevesimmons]

> why in 2019 is there no good way to visualize a decision tree?

You should try https://github.com/parrt/dtreeviz. From Terrence Parr and Vince Grover, released in Sept 2018.

There is a good background article on the problem space and their design iterations here: https://explained.ai/decision-tree-viz/index.html

[autokad]

thanks for the info, but as I said above anything that relies on the installation of graphiz is extremely hoaky, and as you can see on many posts on SO, doesnt work many times.

[tomrod]

I came in to the pydata library after extended time in Matlab/Octave. Learning there first was terrible for programming basics (e.g. polymorphism) but was excellent for ensuring I knew what a given algorithm does/should do.

I highly recommend spending some time in C/C++, Go, or Julia to pydata-first folks that ask.

[jriot]

As a data scientist myself, I learned to write code that utilized arrays and matrices - from the most basic library. From cleaning to analysis to machine learning (I supposed numpy was used for ML). Is this the most correct manner to write code? I don't know but learning to do DS without specific Python libraries has improved my coding ability. Aside from being able to read in different data formats I believe I could do a fair portion of my work in C.

[hikarudo]

What is the pydata library set? numpy, pandas, matplotlib, sklearn and the like? My google fu is failing me...

[natalyarostova]

Yeah pretty much.

[meh2frdf]

Data scientist is a term similar to Web Master, I don’t think it will be a thing in 10 years.

[draw_down]

Someone has to make the tools that are used to automate the marketing, analyze the business data, whatever the task at hand is

[austincheney]

It is like exercise... you are shaping your brain to the problem. The US military actually tests for this in what's called the Army GT score of the ASVAB test.

When I took the ASVAB in high school I scored a 107. That score is too low to become a warrant officer so I had to retake it a couple of years ago for my officer packet and I scored a 129 out of a maximum 130. That puts me in the top 0.1% of testers. I am not smarter or more intelligent than when I was in high school. I do write software though. Every couple of years I look back on my software and algorithms realizing how I continue to improve and see the solutions more clearly.

https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...

[NicoJuicy]

This

The best teacher has been helping collegues. I have been programming a lot better ( less errors and even without sometimes running the application when I'm pretty sure), because I think and analyze more upfront than I used to.

Some things come back, but it's rarely related to me ( eg. last moment spec changes).

I do have to watch out, I notice that basing my code on someone else is ok. But, they always have faulty code in hard to test areas. So make testing easier on things that are hard to test is my next motto.

Also, helping others is a pretty huge timesink :(

Ps. Being in the zone does wonders lately

PS2. There was another threat about videolan yesterday. And nobody heard the entire story about https. I gave the VLC developers the benefit of the doubt, not knowing everything off their infrastructure. . A lot of the comments here on HN disagreed with me ( stole silently upvoted though).

Today I saw a blog post about why..

It was infrastructure based...

I can't understand why I was practically the only one with another view in the subject in this community, where developers come together.

FYI:

Comments are in my history. Mostly on the videolan topic. It's all recent

[SZJX]

I would say that it's the same for the other trades as well though. For example, people might imagine that learning a new human language is trivial with the proliferation of misleading advertisements such as Duolingo or Babbel, but to really learn a language the effort needed is tremendous and it requires constant repeated practice over a long span of time.

The question the author posed was why programmers are paid that much even when some other paths could seem "harder", which seems valid. Sure not all careers are supposed to be "harder" than programming, but they're not as easy as one'd imagine either.

Though yeah at least for now I don't see the situation abating much. The demand is still going strong. Once the proverbial "flood" of the market happens from new grads, things might get worse. But still if you know what you're doing, you know all the right concepts and skills, you should be able to stay on top of the game. There has always been a saying that the irony of the CS degree is that many people who graduated with the degree can't program, while many who can program didn't need to do a degree at all. I doubt the influx of students trying to study CS would change this situation much. Coding bootcamps have been around for a decade yet they don't seem to change the market equilibrium that much.

[criddell]

> they program to accomplish other tasks in their domain of expertise

My kids have mentioned that they might be interested in a degree in computer science because and I've encouraged them to combine that with a second area of specialization. Programmers are everywhere, but a programmer that also knows chemistry or biology or economics or art history or just about anything stands out.

[ummonk]

>Programmers are everywhere, but a programmer that also knows chemistry or biology or economics or art history or just about anything stands out.

Really? I have a physics degree with some experience in rocket science, but my most valuable skillset (measured by how much pay I can fetch for it) is plain old software engineering. I don't think I'd be able to leverage my area of specialization to exceed or even match what I can get from FB/LI/G as a generic software engineer.

[criddell]

You don't think your broader education and experience makes you a better software developer (that encompasses programming, writing, and working with other people)?

[ummonk]

That's a good point. Not going through the standard CS track and straight into software engineering has probably given me broader experience and skills (both soft and hard) that I wouldn't have developed as well otherwise.

[sjg007]

Maybe. I studied CS and bioinformatics. You end up competing with both pure CS folks and also the bio folks that are bioinformaticians. Still I generally agree that some domain expertise is helpful.

[devoply]

If anything good programmers are underpaid. If you are making your company a mil, you should be getting 1/2.

[jjeaff]

But what if another programmer is willing to make me a million for only 25%?

That's how markets work. You get paid what what the market will bear, not what you "should" make.

[est31]

Except if the other side of the market teams up. https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...

[matz1]

That is hard because you have to convince many other people to agree. It's easier for me to just undercut other people by taking less pay. I can for example going remote and live in low living cost area.

[est31]

On the side of the employees, unions have proven themselves to be good means to improve the situation for employees. Here in Germany they definitely have helped in many industries.

My comment was more about the companies though which may form cartels to drive down employee wages. Companies forming cartels is illegal, while unions are legal in many places.

[esoterica]

If unions are the best way to improve compensation why do un-unionized FAANG employees get paid so much more than unionized developers in Europe?

[username90]

Unions are basically legalized price fixing. What happens is that the union negotiate a "fair" price, and then all companies decide to pay no more than said "fair" price. See for example (original is in Swedish):

https://translate.google.com/translate?sl=auto&tl=en&u=https...

[devoply]

As I said good programmers are underpaid. They should figure out how much they are making their companies and ask for more. The market can often afford to pay more, if you just negotiate better. You can also unionize to get your employers closer to what you are worth to them rather than what they are worth to you.

[ken]

> just negotiate better

In every other aspect of computers, the industry has finally embraced usability as a desirable goal, and not just for end-users.

On my first computer, you had to read a 100-page user manual and learn exactly what commands to type. In my first programming language, you had to manually allocate (and worse, deallocate) memory. With my first database, we used to have to go type VACUUM regularly. None of these is true today.

Yet even though some of the highest paid people in the world are members of unions and have agents to do their negotiating, programmers seem to have latched onto this idea that if you're not making top dollar or have your ideal working conditions, you should "just negotiate better".

Why stop there? Tell programmers they should "just program better", too.

> You can also unionize

Have you ever organized? I don't think you realize how difficult this is, especially without strong support from an existing union. There's a reason unions heap rewards on people who do it.

Existing unions also have great labor lawyers. A common response to even thinking about unionization is getting fired. (That was in the news recently because it happened 4 weeks ago here in Seattle.) Labor laws aren't what they once were, and there's usually no consequence to the company for firing organizers.

[tluyben2]

> On my first computer, you had to read a 100-page user manual and learn exactly what commands to type.

Flipside: I can still write software for my first computer without looking anything up, over 30 years after reading those 100 pages. I still know the memory layout, opcodes, assembly etc by heart and it is still the best way today to program that particular computer (which still works in my man cave) today. Yes, today it is all simpler, but the 100 page example I find a plus, not a negative. Maybe you were referring to something but my 100+ page manual was usage and at the same time programming (using was programming beyond the basics) as that was the only way to use the system.

[ams6110]

People who make enough to have agents negotiating their salary (famous actors, professional athletes, other celebrity types) are usually looking at an order of magnitude higher compensation than even the best software developers get. At the lower end of the spectrum (lesser-known actors, musicians, etc.) agents are known for enriching themselves as much as helping their clients. They are just sort of accepted parasites on the way compensation is handled.

[pmiller2]

Suppose I’m working on pulling out some functionality from a large, monolithic application. How much am I making my company?

[jsjohnst]

Depends on what the outcome is. If it makes the site 50% more performant on 25% less hardware, pretty easy to swag it. Same if the outcome makes developers on the team able to ship new functionality 20% faster with 33% fewer bugs.

[thecrazyone]

This seems awfully contrived.

Issue 1: It's very difficult to tell if your contribution got 50% improvement in performance because there were 10 other devs pushing in features and bug fixes. This is the attribution problem

Issue 2: This happens over time. It's very unlikely that your 50% improvement happens every year or month. Because, think for your self, this is compounding with large rates. It grows quickly. 1.5x improvement in 6 cycles (months or years) is 10x. This essentially is the time problem

Issue 3: even if you deliver the results you did, in a large company there's a large bureaucracy and no one person has the ability to increase your salary by that much. This is the control problem.

[nightsd01]

The problem with this argument is that programmers don’t work alone in a vacuum. How do you account for the support staff? The recruiter that hired you? The cleaning lady? The DevOps people? And so on.

It’s avtually fairly non-trivial to be able to say with even a modicum of certainty how much value a given developer brings to their company.

[wffurr]

It's how markets work when wealth is distributed incredibly unevenly, and a weak social safety net makes it intimidating to work for yourself instead.

In other words, markets work that way because that's how the bosses and capitalists want it to work, and have so far been successful at thwarting attempts to use the government to change things.

[someone454]

The simplest and most accurate statement is that it’s an emergent result of the principles of capitalism combined with human nature and not a plot to keep us down.

[WalterSear]

It's both.

[nouveaux]

I'm all for programmers getting paid more. However, by your logic, if the company is losing money, should programmers contribute from their own pockets to keep the company afloat?

Starting your own company (not self employed contractor) gives a really good perspective on what it means to be owner and employee.

[Jach]

How is this his logic at all? The logic is more similar to a sales person. When a company starts losing money, it doesn't try to claw back commissions from its top sales agents to keep afloat. It might lower %s / do layoffs / something else, but money paid is money gone.

The logic is that programmers tend to produce far more value than they capture -- so that gets captured elsewhere, a lot of it typically by management. Except the value can be hard to quantify when the company is old and so is the software, how much of the value is employee #3701 making fixing a bug that's making the product not work for one customer in one instance vs employees #107, #85, and #150 who in their past team's life created the original version of that system to begin with to make the new customer even consider using it? There's no point to moaning about how much you "should" get paid. Just ask for more if you feel underpaid, but be aware that because of competition and because people usually want to hear what more you'll do to justify it, you're not going to always get it.

[devoply]

No they should quit and find another company.

[seem_2211]

You can't keep a significant portion of the upside without taking on more risk.

[devoply]

> Privatizing profits and socializing losses refers to the practice of treating firms' earnings as the rightful property of their shareholders, while treating losses as a responsibility that society as a whole must shoulder, for example through taxpayer-funded subsidies or bailouts.

[friedman23]

which software businesses have been bailed out by the government?

[seem_2211]

Not really related to what we're talking about here though?

[EpicEng]

And in tough times are you taking a drastic pay cut, or jumping ship? Sounds like you want a huge share of the good and none of the bad. Also not sure why you value programming so far above all of the other activities it takes to make a successful product.

[elorant]

Define "making". Does the program/service sells itself? If so, then your argument may stand. Nine out of 10 times though it doesn't. There are other people involved in making the sales and they also require a piece of the pie.

[simonh]

The usual ratio for top flight talent in the finance industry is 10:1. A trader makes JPMorganStanleyofAmerica $5m, they get $500k compensation. That seems reasonable because there’s a direct line between talent and profit. Even so someone had to build the business, capitalise it, create the business opportunities and relationships, train and support the trader and assume all of the risk.

How much of the business risk of the enterprise is your top flight programmer assuming? Are her decisions the only ones that make any difference to the increase in profitability as a result of her work? How direct is the line between back room engineer, no matter how good and profit?

The only case where 50% or near it makes sense is for a founder owner who is also the lead talent. Maybe. Because then they are also creating the business opportunity and assuming a big chunk of the risk.

[vectorEQ]

if you solo develop (usually not the case) an app for 1 mil then make it yourself if you want that large a cut...

[cobookman]

Sales people don't get anywhere near that kind of commission.

Most SWEs I know are not making more than they bring in profit.

[walshemj]

Good ones do - the Oracle one that sold a large company I used to work for a company wide licence allegedly retired on that single deal.

[zappo2938]

It's a team effort. I stopped trying to get software projects off the ground by myself

[jenscow]

Sarcasm works really well in a text-only format!

[mathdog]

> Can this last forever? Of course not, nothing lasts forever. But wondering why the wealthiest corporations in the world pay their workers high salaries is perhaps like wondering why water is wet. Software has a low marginal cost, and the rest is basic incentives for the corporations.

Agree