[lame88]

A constantly cycling proliferation of different languages, frameworks, libraries, etc. that all do the same things in a different way and are most often mutually incompatible with each other and have entirely different ecosystems with their own comparative advantages but also major pitfalls. This causes tech workers’ investment in skills to get more out of shallow knowledge and trivia than on deeper concepts, creates silos of employment opportunity based on the trivial knowledge workers have, and hinders the ability for the software engineering field as a whole to have a large pool of shared knowledge and develop and evolve stable, relatively timeless systems and tools of high quality, both as end products and in intermediate tooling toward those ends.

[closeparen]

Learning new skills is easy enough, what kills me is the enormous manpower invested in churning working code.

Easily 30% of my company's total engineering effort is just treading water, migrating from a deprecated platform to another one that will be deprecated by the time the migration is complete. Typically because the original team's standard 18-month tenure has elapsed and the new guy was under-leveled at hiring so he needs impact for promotion.

It's a great disservice to our field that people so deep in the stack are so comfortable changing their minds all the time. The Python 2.7 thing feels like the Library of Alexandria. Burning down mountains of perfectly good working code just because we can.

Backwards compatibility is tragically underrated.

[zelly]

Backwards compatibility causes a lot of evil. I would prefer that breaking changes come with automated tools to migrate. Sometimes bad decisions get made, and we shouldn't have to carry the burden of that forever out of laziness/stubbornness. What ends up happening is the opposite of what you wanted, because after enough backwards compatibility debt adds up, someone starts a fresh competitor that takes over.

[gridlockd]

> I would prefer that breaking changes come with automated tools to migrate.

The hard part of the migration isn't that you can't automate it. The hard part is that you can't automate verifying that it didn't break anything. So you rather stick to what you have.

> Sometimes bad decisions get made, and we shouldn't have to carry the burden of that forever out of laziness/stubbornness.

Here's the deal, Mr. Developer who wants to change everything because the old stuff is somehow bad and the new stuff surely is better: I almost certainly have better use for the money than to spend it on migrating the code. The old code will keep working. That's not laziness, it's prudence.

Developers always exaggerate the benefit of rewriting stuff and changing things around. I understand why, you'd rather arrange the code you work with to your taste than whatever horrible stuff is there already. The problem is that every other developer feels the same way, but with a different taste.

For instance, some people hate object oriented programming, some people think procedural programming is somehow bad, some people believe functional programming is generally good. I believe the best programming is the one you can just stop arguing about and solve the damn problem with, sooner rather than later.

> What ends up happening is the opposite of what you wanted, because after enough backwards compatibility debt adds up, someone starts a fresh competitor that takes over.

That's not true at all in the vast majority of businesses cases. The benefit of the migration would have to be so spectacularly high, it would have to offset its cost. Again, this almost never happens.

Whenever you break compatibility, you burn good will. Whenever you break compatibility, you give me an opportunity to switch to your competitor. If I have to migrate, I might as well migrate to something else.

[closeparen]

When code is hard to maintain, extend, or debug we should absolutely refactor it. Even if something is fine but can be expressed better using a feature from the language's new backwards-compatible release, go for it.

The tragedy is mandatory, all-out migration for code that was already high quality, and just happened to be written for an older stack.

[gridlockd]

All code is "hard to maintain, extend and debug" to the person who didn't write it and therefore doesn't like working with it. Developers like to exaggerate this all the time, because they want to maximize their own comfort.

Even if developers were able to objectively asses that code is hard to maintain/extend/debug, that doesn't mean changing it is the right business decision. It may or may not be. From a pure efficiency standpoint, it may well be a net loss. From a developer psychology standpoint, it may be worthwhile.

You also can do a lot of refactoring without breaking a lot of code, especially if you didn't buy into the idea of writing tiny functions and unit tests for every piece of functionality.

[closeparen]

If you’re stuck working with people who think they are the only ones ever to have written good code, I’m sorry. That must be unpleasant. But please don’t take that out on everyone else who merely gives a shit about the quality of their work.

Most of the code I refactor is my own from a few years ago, btw. Usually because I learned new information, past choices turned out to have regrettable unforeseen consequences, etc.

But we have learned to be really careful about public APIs. Implementations may change, interfaces are forever.

[TsomArp]

It doesn't matter if it is easy. If you have to waste time because the language-of-the-week came, which multiplies for N number of devs, is still a huge waste of time.

[Viliam1234]

I was thinking about the reasons why this happens, and I see two major factors:

1) Tension between abstraction and optimization. To put it shortly, abstraction is about ignoring the details, optimization is about fine-tuning the details; you can't do both at the same time. Which is why different programming languages make different kinds of compromise. You could make a beautiful language or framework with elegant abstractions, then look at the performance and cry. Or optimize for performance, and then cry while reading and debugging the code.

2) Tension between mathematical elegance and the cost of hiring people who are great at math. Some developers care about having the code elegant from mathematical perspective, but the companies optimize for having a product to sell, as cheaply as possible, which involves a lot of cutting corners. A product full of bugs can still make you millions of dollars. And what's the point of having a mathematical proof of correctness of your current code, when the business requirements are going to change tomorrow anyway.

[aabbcc1241]

For the tension between abstraction and optimization. Red, the language claim to be fullstack, from hardware driver to high level programming for GUI. It can be used for performant system programming as well as high level functional programming and meta programming.

For people interested by this rebol inspired language https://www.red-lang.org/p/about.html?m=1

[Viliam1234]

I am not familiar with this language, so I don't know how (or whether at all) it addresses the problem I was thinking about. Let me try to explain it:

Let's take the simple concept of "array" or "list". Mathematically speaking, it's a very simple concept: integer numbers are mapped to objects.

But there are so many ways to implement this mathematical abstraction. Do you assume the integer numbers will be used in a sequence, like 1, 2, 3, 4, 5; or completely arbitrarily with possibly large gaps, like 1, 1000, 1001, 5000? Will it be accessed from different threads? Will it have a "create phase" when it is constructed in a single thread, followed by a "use phase" when it is accessed from multiple threads but read only? Will it be modified frequently, or rarely, compared to mere reading? Will you need to make copies of it for further independent modification? Etc.

Now, one possible way is to choose one specific answer, and make the standard "list" in your language mean exactly that. For some purposes it will be okay, for other purposes it will suck, and someone will create a library providing an alternative implementation; and users will complain why they need an extra library for something that should have been part of the language.

Or perhaps you will provide multiple implementation, and users will have to choose. And they will complain about this being too complicated.

You might also spend years trying to find one perfect implementation of "list", that will fare relatively well under all circumstances (never the best one, but also never the worst one).

It would be nice to have a language where the programmer wouldn't have to worry about performance of the underlying data types. Just use them as mathematical abstractions, and everything will work fine. Like, you would still have to worry about algorithmic complexity of your code, but you would not have to worry about accidentally using the existing stuff in a wrong way which is not obviously wrong (and would not be wrong for a different implementation).

This is different from merely allowing people to use both high-level and low-level concepts in the same language. This is about how to implement the language in a way that allows you to do high-level as much as possible, without suffering terrible performance consequences. And I don't mean consequences like "this will be 100 times slower", but rather "this implementation of list, when used in this specific way, will actually have exponential complexity where some other implementation would have been polynomial". Because ultimately each implementation has a weakness, and one must be chosen. And if you treat other pieces of code as black boxes, it means you never know whether you made a good choice.

[aabbcc1241]

You're right, it's not just about the language, also the 'built-in', 'ecosystem' and 'foreign interface'.

[Psyladine]

Nevermind all that, even starting arrays at 0 or 1 will break the momentum of the mightiest dev waves.

[hanoz]

And conversely, the problematic consequence and cause of all that: Resume Driven Development.

[zelly]

You don't actually have to invest in these new shiny ecosystems. It's just a choice you made. There are still people making a living on Fortran, C, C++03, Java <5, Angular. All of that still works on modern hardware and will for the next 100 years. You just probably won't be in a coastal place with pool tables and free food.

[mamcx]

I think this is a manifestation of a more bigger issue: C, C++, JS.

This happend because our foundations are shaking, and is requiere to put on top some sanity. But nothing can work because: This industry not let go of the old and bad (C, C++, JS, Bash, etc) and refuse to do a significant change.

Too much change is bad.

Too little change is bad.

Have both things at odds in a fierce battle is worse.

[fmjrey]

Totally agree, this is a problem I also see plaguing our industry, and solving it is indeed a priority. We developers or engineers like wasting our time in abstractions, i.e. virtual worlds of isolated castles and babel towers of different languages and expertise. In such virtual worlds we can create and fantasize on all the abstractions we want, be the expert, play god, decide on which constraints matter, and build a world based on that. Such is the power of the non material software world indeed. Who does not want to be god? To caricature I'd say a significant part of the IT industry is about writing virtual entertainment: video games to be sold to the wider public, and toys developers and engineers can play with: languages, IDEs, ecosystems, frameworks, etc.

Earlier in this thread I wrote about data being the top priority [1], the raw material, that we rarely give data the primary focus it deserves and instead focus too much on the processing side. More concretely: dashboards showing instrumented processing clusters give a biased view that does not focus on what matters at the end of the day. What we also need are dashboards showing data flowing between sinks and sources, data quantity, data quality, etc. Sure resource utilization and efficiency matters, but only after we can validate we still have the right output, and that input is of proper quality. If output contains garbage, is it bad processing or is it garbage from the input? And if something is wrong with processing, do we know the impact downstream? In other words instrumentation should include data sensors, not just processing sensors: data counters, validation points, invariants, etc. Because at the end of the day, when the power goes off, do you know what's left to recover? Do you prefer your customers telling you about an unfulfilled order or do you rather want it to be detected earlier? If you get audited for GDPR, do you have a map of your sensitive data? In terms of security, is it about protecting clusters and containers, or is it about protecting the data? Once you get the data side right many things become simpler, but if you get it wrong, as we often do, we create a world of problems. Giving data its proper place in our engineering practices will certainly change our industry for the better and bring it closer to "reality" with less danger of veering into the virtual for the sake of it.

In a world where software is increasingly involved in human activities this would have a great impact. However I don't think we should stop there, as I believe this is part of a larger trend I'm concerned about: the idea that, not just in software development but in most human endeavors, we're increasingly favoring spending time in virtual/man-made spaces and activities, at the expense of the real world, the place and time we're at, nature and the environment. As if we want to escape the physical conditions we're in: whether it's our body, our environment, society, the work we do, etc. When one can't see a way to influence the real world a tendency would be to start operating in a virtual one where we get the illusion to have an effect, make some money, be an expert, etc. Oh and let's not forget this desire to put as much tech between us and the real world, as if we don't want to experience it directly, it's too icky, and instead need devices to offer an indirect perception: wearable tech, navigation by GPS, remote controlling tractors in vast industrialized agricultural fields, etc.

A friend working in a supermarket chain told me this story: he often advised a younger manager to consider better maintenance procedures of their A/C system, but to no effect. Now that my friend is retiring soon this younger manager is proposing to make an excel chart to track energy consumption in order to optimize setpoints, and asks my friend for his approval. Really? Approve perception to be limited to an excel chart? Isn't that the same as leaving the windows open and wanting to change the setpoint?

[1] https://news.ycombinator.com/item?id=22277875

[gdsdfe]

Do you work in software? It's making new things with new things that software makers better at what they do.

[lame88]

Sounds nice. And idealistic. The reality is at this point the newer is often not much better, or may be just a step sideways rather than forward, and therefore does not replace anything, but rather siphons off a portion of the investment in the pool of alternatives around it. The benefits tend to not justify the costs associated with skill fragmentation and time and effort needed to learn them. More concretely, im not talking about jumping from C to Java. Im talking about things like the thousand different web languages that dont need to exist at all.