[Silhouette]

The fact that you can have different coding styles and APIs in your project and it will work is proof that it is backward compatible.

I don't think anyone is disputing that React is still backward compatible with the old API. The point is that there is a difference between having two APIs that both still work and having one stable API.

Backward compatibility is implied if your API is stable, but that's only one aspect of stability and there are many other practical questions. Will documentation and tutorials from a few years ago still be relevant? Will someone joining your team need to learn anything different before they can be productive? Will other libraries you have used previously that integrate with React still work with your new code? Will new libraries you might want to use today that integrate with React work with your existing code that uses the old React API?

Having multiple integrations with different APIs or different libraries to do the same job is a form of technical debt, and sometimes it can become quite expensive.

I find complaints about the stability of JavaScript kind of ridiculous. Everybody is literally still transpiling down into ES5 (2009) and most only dropped IE11 this year (2013), the main libraries/frameworks tend to make expert use of semver, polyfill old APIs and release codemods to automaticallly perform syntax upgrades.

Well, there are a lot of things you've said there that aren't quite the whole story.

A lot of modern build processes don't just target vanilla ES5 any more. They'll target something like the most recent two versions of the evergreen browsers and any specific minimum versions of other browsers they need for their particular market. This produces more efficient production code because all major browsers now support much more recent JS natively.

There are also some modern JS APIs that you can't completely polyfill. You have to target a runtime environment that provides real support for those.

Semver is great when it works, but when your culture is to have absurdly deep trees of nested dependencies with hundreds or even thousands of indirect packages involved, things still break often enough to notice.

Major libraries might provide automatic codemods for straightforward, unambiguous changes. However, sometimes you can't divine the original developer intent automatically. Then some manual intervention is required.

I'm interested whether there are any programming language ecosystems that have done this better? As far as I can see Python 2 to 3 caused hard rewrites, etc.

Almost all of them, in my experience. JS, including the ecosystem and culture around it, has by far the worst stability story of any major language I have used.

This is partly a cultural problem. Writing code to last simply isn't part of the mindset or skill set of many people working in web development.

Many tools and libraries we rely on for the most vital functionality are FOSS with at most a small team working on them or maybe just one single person. That means even really successful and widely used projects can easily fade away as those behind them lose interest.

Much of what is built on top is written by people being paid astronomical amounts of money who will job hop in less than two years for even greater astronomical amounts of money. They might make it to senior/staff levels a few years into their careers or maybe become the CTO at a startup, never having either built a new product from scratch or maintained one for 5+ years to see where those decisions they made earlier on come back to haunt them. Then those people are the ones leading and mentoring the next generation of newbies working in the JS community, and the cycle continues.

It's also partly a technical problem. The language doesn't provide much to help with building large-scale software and maintaining it over the long term. What it does provide is relatively recent. The tooling is relatively weak, well behind the state of the art for many other popular languages, which in turn lag the more research-focussed languages where radically different ideas get explored.

I believe a big factor in these cultural and technical problems is the way that you can just change your code and push out a new version any time you want since you have complete control over the servers hosting it. It has also become the norm to depend on online APIs provided by other parties who can do the same. This means concepts like managed releases and support lifetimes and stable interfaces are given less importance.

In striking contrast to all of that, there is code I worked on professionally a couple of decades ago that was written in C and would still compile and run on any modern platform with few or no changes. Likewise, there are professionally developed libraries in ecosystems like Java and .Net, which are platforms widely used for large-scale enterprise software development, that have been around for a decade or more and are still as active as ever. Even in Python, the 2-to-3 shift took place over about a decade, taking the community and the tools and libraries along with it. The language is still mostly compatible with Python 2 code, and there were tools to mechanically update code for most of the main breaking changes, so many of the popular libraries from the Python 2 era are still available and working fine with Python 3.

The only other language I've used where instability was as awful as JS world is Haskell, and that's obviously not a mainstream language in the same sense. It's mostly used by a community who are very open to trying big new ideas and changing how they work frequently, and that's part of its attraction to those people and also probably a reason that ideas from Haskell sometimes drift into more popular use over time but the language itself remains an eccentric choice for production use.

[lhnz]

  > The point is that there is a difference between having
  > two APIs that both still work and having one stable API.

This is the direct effect of high levels of backwards compatability. You can't change APIs; you have to create new APIs and then deprecate previous APIs. That's the only way of avoiding a breaking change (other than never changing something and endlessly layering leaky abstractions on top of it).

  > Having multiple integrations with different APIs or
  > different libraries to do the same job is a form of 
  > technical debt, and sometimes it can become quite 
  > expensive.

I disagree. If you don't need to do any work to something it has zero cost. Write tests to its behavior and it can be a blackbox to your team until you truly need to deal with it.

  > A lot of modern build processes don't just target
  > vanilla ES5 any more. They'll target something like 
  > the most recent two versions of the evergreen browsers
  > and any specific minimum versions of other browsers
  > they need for their particular market.

This isn't mainstream yet. Until very recently the majority of companies were targetting IE 11. A lot of companies changed this in 2020 (Microsoft, Atlassian, etc), and additionally due to Node finally supporting ESM and the release of Snowpack and Vite, many open-source developers are finally targetting something greater than ES5. But you're wrong to imply that it's the norm.

  > There are also some modern JS APIs that you can't completely polyfill.

People tended not to use these due to needing to support older browsers. But sometimes APIs could be partially polyfilled to an extent that was sufficient enough to use them (like `Proxy` with `proxy-polyfill`).

  > Almost all of them, in my experience. JS, including the ecosystem and 
  > culture around it, has by far the worst stability story of any major language
  > I have used.

OK, yeah, I'm not seeing this at all. For a start, TC39 has rules against breaking the web (see "One JavaScript", etc: https://tc39wiki.calculist.org/about/faq/). It's due to this choice that we still have mistakes like `typeof null === 'object'`...

Also, I don't think your opinion has wide acceptance within the industry. See comments by Kyle Simpson (https://twitter.com/getify/status/1195362162857910273) or even Yaron Minsky decrying their inability to fix old mistakes (https://twitter.com/yminsky/status/1067926815992410113).

Of course, TC39 is not perfect and there are very rare occasions that minor incompatabilities are introduced (https://tc39.es/ecma262/#sec-additions-and-changes-that-intr...) but "the worst stability story of any major language" is quite an absurd statement if you consider the minutae that has made that list.

  > [...] Much of what is built on top is written by people being paid astronomical amounts 
  > of money who will job hop in less than two years for even greater astronomical amounts
  > of money. 

This is all true. But most of the products these people create are also being replaced every 2 years so it's not as big an issue as you think it is. We're able to go back to projects written in JavaScript 10+ years ago and they work in newer browsers, but the majority of projects are thrown away or rewritten for completely different reasons before this point (normally employee attrition on smaller projects is enough to lose the knowledge required to fix bugs or add features and so companies rewrite to get back that understanding).

I guess a form of backwards compatability we don't have is the ability to upgrade a tooling dependency through multiple breaking changes and not find out that the configuration system has been completely overhauled. That is unfortunate. But the code will build/run when you pull it down the first time from your repository. If you then decide to make upgrades to use new tooling you have to know what you're doing -- tooling upgrades are confusing.

  > The language doesn't provide much to help with building large-scale software and 
  > maintaining it over the long term. What it does provide is relatively recent.
  > The tooling is relatively weak, well behind the state of the art for many other
  > popular languages, which in turn lag the more research-focussed languages where
  > radically different ideas get explored.

I disagree. It has testing frameworks, it has automated code formatters, it has linters, it has type checkers, it has tools to automate rewriting 10,000s of lines of code via the AST, it has tools to automate producing changelogs, it has tools to visualise dependencies, it has tools to improve the performance of scaling a monorepo, etc. Some of these tools are significantly worse in other programming language ecosystems or do not even exist.

  > Even in Python, the 2-to-3 shift took place over about a decade, taking the community
  > and the tools and libraries along with it.

Are you trying to tell me that a breaking change that was so bad that it took almost a decade for the community to complete is something that we should aspire to?

I don't think that JavaScript has serious backwards compatability issues at all. Certainly not in the language. Generally not in the major libraries (React, etc). It perhaps has some issues in breaking changes to the tooling which could make upgrades very confusing (e.g. Babel changed how configuration worked a few times and this was painful due to obscure error messages).

On the other hand, I will say this: it is true that the open-source JavaScript community is high on modularisation, scrappy/innovative and prone to releasing something for fun and then ghosting their consumers. In situations where the quality of the package was high enough that you don't need to change anything this presents no issues, but sometimes when you've used a package with a broad vision that the creator has later on bowed out of fulfilling or which has later on produced lots of bugs, that then leads to engineers swapping it out for newer, shinier or better maintained (for now) things... That is the 'stability' issue that we tend to have issues with and it's the reason that I (and a small minority of other people) have started to 'vendor' or inline code into our projects. As Tom MacWright recently wrote "One of my golden rules is that you shouldn’t blackbox things you don’t need to. I like to “use dependencies for efficiency, not ignorance.”" (quality article by the way: https://macwright.com/2021/03/11/vendor-by-default.html).

[Silhouette]

This is the direct effect of high levels of backwards compatability. You can't change APIs; you have to create new APIs and then deprecate previous APIs.

I don't see how that follows at all. You can't make breaking changes to APIs, but it's fine to extend them or even expose different levels of abstraction as long as the underlying model you're using stays consistent and you provide suitable defaults where necessary.

If you don't need to do any work to something it has zero cost.

This is tautological, but if we're talking about a typical web application then you are working with that code, every time you make a new build. It might sit there as an untouched black box for a while, but if it has any kind of external dependencies or it relies on anything that could change about the language, platform or other parts of your own code, then there is always a risk that it will later break, and then you'd better hope you have someone around who still knows how to fix it.

This isn't mainstream yet. Until very recently the majority of companies were targetting IE 11.

You've made several claims like that, but in the absence of data there's no way to know whether you're correct. For example, preset-env has been in production for 3+ years now and had an extended beta/RC phase before that. It's true that it defaults to ES5 if you don't configure it, but the Babel docs explicitly recommend that you do, and it is literally a one-line change in your package.json to do something more modern and efficient based on Browserslist unless you genuinely need that degree of backward compatibility.

Of course, in some cases, even quite recently, you did need that. A relatively small proportion of business customers stuck in the IE11 era could still represent significant revenue at risk if you dropped support. But it will take more than an unsupported assertion to convince me that this has been the norm for most web development for quite some time. At one of my businesses, we've had literally no-one using IE11 visit a B2C web app for years. At another that mostly does outsourced development work, we've excluded IE support contractually for a long time and no client of any size has pushed back in years. There must be many thousands of other small businesses out there like mine, and I have to assume that they have senior devs who sometimes read the docs for the tools they use too.

OK, yeah, I'm not seeing this at all.

I think that's because you persist in interpreting my comments as being about JS itself, when I've tried repeatedly to emphasize that it is the culture and surrounding ecosystem that are the bigger problems. I'm not talking about the technical committees defining the language. I'm talking about the developer experience of using it in production.

On that score, I stand by my earlier comments. After many years of programming professionally, using many different languages, I have yet to find any more flaky ecosystem than JavaScript's other than the deliberately fast-paced world of Haskell. Nothing else even comes close for how much wasted effort I've seen over the years just keeping all the plates spinning so nothing falls off and breaks.

But most of the products these people create are also being replaced every 2 years so it's not as big an issue as you think it is.

Yes, I understand the commercial argument. The bizarre economics in the web dev industry sometimes make throwing entire projects out and rewriting every 2 years affordable. That doesn't mean it's not a horribly unstable environment, which was the original point of contention, nor that disregarding traditional wisdom about the dangers of big rewrites isn't a bad idea. It just means parts of the industry are so rich that they can afford to have a toxic culture where quality is a second-class citizen and the only answer to producing so much unmaintainable junk is exactly what you said. It's much nicer working in other parts of the software industry where what you're producing is expected to work and to last, but sadly it doesn't always pay as well.

It has testing frameworks, it has automated code formatters, it has linters...

And these kinds of things are advanced tooling, in your view? We had those in our IDEs for building desktop software back in the last century. Except they were automating rewriting millions of lines of code on 2000-era PCs, not 10,000 lines of code on 2020-era PCs.

I'm not sure what other popular programming language ecosystems you think don't have these kinds of tools today or have tools significantly worse. But in web development, TS is the current hotness because the developers finally realised that explicit typing is helpful for building and maintaining more than small systems, and not so long ago build tools adopting tree shaking was being heralded as a great advance for reducing the size of builds, and before that we got the radical new concept of a module import/export system so you could write code in more than one file without jumping through crazy hoops, and before that we discovered that immediate mode UIs and declarative specs are a thing. All of these ideas were well known and widely used elsewhere in the programming world much, much earlier.

Are you trying to tell me that a breaking change that was so bad that it took almost a decade for the community to complete is something that we should aspire to?

It took that long because that backward compatibility and stability we've been talking about meant a lot of older projects didn't need to move any sooner. And yes, having a platform where you can write code and it keeps working and the ecosystem still positively supports it for another decade is definitely something we should aspire to.

Generally not in the major libraries (React, etc).

Don't tell that to anyone who uses React Router. Or who builds their app with Webpack. Or Babel, as you mentioned. And again, it's important to note that you only said backward compatibility here. If we're talking about the broader concept of stability then you also have to include high profile libraries like Moment giving way to more modern alternatives because although they still work, they don't lend themselves to modern programming styles or play so nicely with other tools.

These comments are getting long and I suspect we may have to agree to disagree on some of this, but if you haven't worked much on software in other parts of the industry, I encourage you to look at what is considered the norm and acceptable practice in fields where better quality and greater longevity are more highly valued. If we built security libraries or the embedded software controlling industrial machinery the way a lot of online companies build their web apps, we'd be lucky to survive the night.

[lhnz]

I'm trying to respond in good faith, even though I feel you unnecessarily resorted to trying to condescend me.

You wrote:

  > there is a difference between two APIs 
  > that both still work and having one
  > stable API.

As I explained, "this is the direct effect of high levels of backwards compatability. You can't change APIs; you have to create new APIs and then deprecate previous APIs."

I thought my point was obvious, but will explain it with more words:

If never making breaking changes to APIs is a strict requirement, all API changes must be additive. That means backwards compatibility entails the existence of more than one API to do the same thing. It seemed like you were insisting that there should be "one stable API" that never needs changes instead of "two APIs that both still work" and this is unrealistic for something that has existed and responded to changes in use for ~25 years.

  > if it has any kind of external 
  > dependencies or it relies on anything
  > that could change about the language, 
  > platform or other parts of your own 
  > code, then there is always a risk that
  > it will later break

This is a useful point. For dependencies that never make breaking changes you will be OK. But if the dependencies you have are likely to change and likely to be upgraded elsewhere in the codebase and potentially might make breaking changes, we should consider them a risk to the project.

I agree that this should be highlighted -- but, in the context of what you were talking about, which was with regards to code that uses old features of React, you are incorrect. As I mentioned to you, compatibility packages exist for all old React lifecycle methods and deprecated patterns. Therefore, by design, you are not being forced to upgrade the patterns in old parts of your codebase.

I am trying to make specific claims about concrete things that I can evidence. The existence of compatability packages for features that have been deprecated for 5 years invalidates a specific argument about needing to upgrade old React code due to changes in the API.

  > You've made several claims like that, 
  > but in the absence of data there's no 
  > way to know whether you're correct. 
  > For example, preset-env has been in 
  > production for 3+ years now and had
  > an extended beta/RC phase before that.
  > It's true that it defaults to ES5 if 
  > you don't configure it, but the Babel 
  > docs explicitly recommend that you do.
  > [..]
  > But it will take more than an unsupported
  > assertion to convince me that this has
  > been the norm for most web development 
  > for quite some time.

If it defaults to ES5 why are you asking for me to provide evidence as to whether companies have generally targetted IE11? It is the default; it is very unlikely that there is a majority of people that are configuring it...

  > But it will take more than an unsupported
  > assertion to convince me that this has 
  > been the norm for most web development 
  > for quite some time.

My claims are the following:

1. ES5 has been the default for front-end packages published to NPM since it was released (https://twitter.com/_developit/status/1289317757146693632). (I'm linking to Jason Miller's tweets since he has been the one most obssessively pushing for this to change. The problem we've all been having is that the `package.json` fields do not make it explicit what syntax a package is using.)

2. IE11 is in rapid decline (https://twitter.com/TheRealNooshu/status/1348955568459968515) and only recently have big corporations dropped support for it (Atlassian (March 2020): https://blog.developer.atlassian.com/retiring-ie11-support-f... ; Microsoft 365 (August 2021): https://docs.microsoft.com/en-us/lifecycle/announcements/m36... ; LinkedIn, Twitter, etc (2020): https://twitter.com/swyx/status/1260627626739130369). As I said "most only dropped IE11 this year". I guess I was off by one year, but I was talking about recent announcements.

For context, this was to evidence my opinion that the JavaScript community has been very slow moving and careful to output code that is executable by old browsers to the extent that we've supported a browser released in 2013 and used a syntax released in 2009.

  > And these kinds of things are advanced
  > tooling, in your view?

Check back to my text and you will see that I made no such claim.

You claimed that "the language doesn't provide much to help with building large-scale software and maintaining it over the long term" so I demonstrated that we have all of the necessary tooling to do so and invalidated this.

  > We had those in our IDEs for building 
  > desktop software back in the last 
  > century. Except they were automating
  > rewriting millions of lines of code [...]

Can you point me towards examples of the usage of AST rewriting tools that are actively being used by open-source library maintainers in other ecosystems to help their consumers do upgrades in large code-bases (e.g. I can see that `clang` has AST tooling, but I don't see examples online of this being widely used: https://devblogs.microsoft.com/cppblog/exploring-clang-tooli...).

Within the JavaScript ecosystem this is quite popular (1. https://github.com/reactjs/react-codemod ; 2. https://github.com/mui-org/material-ui/tree/HEAD/packages/ma... ; 3. https://github.com/emotion-js/emotion/tree/main/packages/esl... ; etc.) Teams that maintain huge codebases have tooling to safely automate their upgrades and the best library maintainers provide their consumers with these AST transformations.

  > Don't tell that to anyone who uses 
  > React Router. Or who builds their app
  > with Webpack. Or Babel, as you mentioned.

I agree that those particular libraries haven't been maintained well (they make extraordinarily large changes in major versions to the extent that sometimes it might as well be a whole new library).

  > I encourage you to look at what is considered the norm
  > and acceptable practice in fields where better quality 
  > and greater longevity are more highly valued. If we 
  > built security libraries or the embedded software 
  > controlling industrial machinery the way a lot of 
  > online companies build their web apps, we'd be lucky
  > to survive the night. 

I think your argument has been a moving target. Originally, you pointed your finger at React saying it has a stability problem, so we showed you that they've been incredibly good at keeping backwards compatibility and that you could upgrade your version of React to regain access to the ecosystem's newest libraries built on top of new APIs without needing to rewrite your legacy code through the usage of compatability packages.

Then you implied that it was an overall JavaScript ecosystem stability problem, so I showed you the level of effort that TC39 and the brower maintainers go to keep backwards compatability. And I showed you that most in our ecosystem have been carefully supporting the lowest commmon demominator users for almost a decade by producing web apps that work in IE 11 and packages that first-class ES5 syntax, and gave you examples of big webapps that only recently broke from this tradition.

Now you are pointing at exceptional cases involving particular poorly maintained libraries and tools, and telling me that we lack tooling to deal with scale (which is a separate concern). If you don't think Webpack and React Router are good citizens with regard to their API stability and this annoys you because you like to be on the latest/greatest then don't use them. I and many others also specifically avoid these libraries/tools and yet don't paint the whole ecosystem with this one brush. (On the topic of tooling, it has only been very recently that we've been getting tooling that can deal with high-scale development, however it is happening now.)

  *  *  * 

I think that possibly what you're noticing is the second-order effect of more and more usages of JavaScript and web technologies causing new APIs to steadily be added to the web with the old APIs remaining forevermore. The original APIs were made to display and navigate documents. As the years have passed more and more APIs have been released to support more complex types of applications and this resulted in overall churn in user-space as library creators iterate to produce better domain-specific APIs. We provide a high-level of user-facing stability despite this.

I don't think it is fair to compare what we are trying to achieve to what is important when controlling industrial machinery. The use cases of that are much more static, while in 2021 people on the Web are creating tools like Figma (https://www.figma.com/), Office 365 (https://www.office.com/), Buerli (https://buerli.io/), and Spline (https://spline.design/). This is very far diverged from where we started and the fact it has been achieved without breaking the web is impressive. It's also quite clear to me that high-quality tools like these aren't achievable without the creation of impressive and scalable tooling (some of which has been released to the public).

OK, let's agree to disagree...

[Silhouette]

I'm sorry if anything I wrote came across as condescending; that was not the intent. You seem reasonable enough, and objectively I agree with much of what you've said. However, I think we've been talking slightly at cross-purposes.

I won't try to go point by point now as these comments have become very long, but overall the difference between our positions (as I intend mine and as I currently understand yours) seems to be that you're focussing on specific technologies like JavaScript or React, while I'm talking more holistically about the entire ecosystem around them.

For example, you have talked about backward compatibility being excellent within JS-the-language, and about React continuing to support its class-based API alongside the functions-and-hooks alternative. That compatibility is not in dispute.

However, when I'm talking about stability, I'm also including everything that is built around JS or around React, the libraries and tools and documentation and people. And in these respects, the JS ecosystem is much faster moving and more fragile than almost any other language I have ever used.

The kind of bifurcation of APIs we've seen with React causes a lot of churn even if the library itself is still backward compatible. If you looked at blog posts or Stack Overflow answers or Reddit discussions about React from as little as three years ago, many of them would have little relevance to the newer API that has obviously become dominant within the community since then. On the other hand, if you hired someone with a couple of years of React experience today and asked them to maintain older code in your system that used the class-based API, they might have no idea what they were looking at. And if you found a useful library to work with React components but it only provides a hook-based API, again you can't use that functionality from your older class-based components. Your old code might still build, but it's essentially unmaintainable unless developers who have come to React more recently first learn what is effectively an entirely different library from their perspective, and even then you're limited in how much you can integrate that older code with more recent contributions to the ecosystem. (This paragraph is an elaboration on the point I was trying to make when I first joined this discussion.)

For the wider JS ecosystem, we've talked about the breaking changes in major tools like Babel and Webpack and in popular libraries like React Router. You suggest that these are "exceptional cases involving particularly poorly maintained libraries and tools". I suggest that there is, unfortunately, nothing exceptional about this sort of abandonment of earlier versions and interfaces within JS world. For one thing, Babel and Webpack aren't just any tools, they are arguably the two biggest names in the JS ecosystem, so when they break, the damage is widespread. For another, they are far from isolated cases. To give another example, Angular was React's big contemporary "competition" and it too essentially abandoned one API in favour of a radical rewrite for version 2, turning any project using the older framework into legacy code almost overnight.

Before React and Angular, we had a string of moderately popular UI libraries built around various levels of data binding. These were must-know technologies in their day for anyone who wanted to work on a project using them, but none remained popular for more than a few years, again leaving all of the individual knowledge and documentation and old code bases largely obsolete.

This pattern isn't unique to UI libraries, either. For example, as JS has evolved and added support for promises and then async/await, many older libraries whose interfaces weren't designed to support that programming style have been left behind, again including some that were very well-known and widely-used.

And with tools, it's a similar picture. Although Webpack is probably the most popular tool for building web app front ends today, over the years we've used shell scripts, self-contained tools like Browserify and Babel, task runners like Gulp and Grunt (with their own surrounding package ecosystems) to co-ordinate the ever-increasing number of tools used in build processes, more comprehensive tools like Webpack (again with a big surrounding package ecosystem) to try to unify everything in one place and bring some structure back, attempts to simplify the horrendous complexity that unification had introduced (the later versions of Webpack and the likes of Parcel), and now we're swinging back towards more specialised tools but in some cases with much better performance such as esbuild, but ironically these newer and better tools may be hard for a lot of projects to adopt because so many people are boxed in by their existing tools and have to wait for someone to provide some plug-in package for those existing tools in order to integrate the new ones. And again, breakage is not unique to Webpack. For example, not long ago, Parcel pushed out a breaking change, and because the effort from the Parcel devs is mostly focussed on the still-in-beta Parcel 2 that naturally has some major limitations of its own, this left teams using Parcel with broken builds either way until they figured out what the problem was and then probably rolled back to an earlier version of Parcel 1.

So I suppose my point is that all of that disruption can and does still happen frequently within the JS ecosystem, despite the backward compatibility of JS itself and the willingness of at least some of the big libraries to continue supporting old APIs after launching new ones. Stability, in the sense I'm using the term, implies a reasonable degree of backward and forward compatibility. The converse, unfortunately, does not hold. A good compatibility story is necessary but far from sufficient for a stable, productive ecosystem.