Hacker News new | ask | show | jobs
by mattbrewsbytes 751 days ago
> RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.

REST API's are a proven solution for the problem of other apps, including front-ends, needing data from a data store. Using JSON is much improved over the days of XML and SOAP. Beyond that there haven't been advancements in technology that cause fundamental shifts in that problem space. There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.

There are so many developers out there that can't stand plugging in proven solutions to problems and just dealing with the trade-offs or minor inconveniences. Nothing is going to be perfect and most likely a lot of the software we write will cease to be running in a decade.

7 comments

REST APIS suck for nested resources. GraphQL is a huge breakthrough in managing them.

Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.

With internal REST for companies I have seen so many single page specific endpoints. Gross.

> There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.

You could argue almost any tech solution in a non-pure tech play is largely in consequentially as long as the end goal of the org is met, but managing REST APIS were a huge point of friction at past companies.

Either it goes through a long review process to make sure things are structured "right" (ie lots of opinions that nobody can sync on) or people just throw up rest endpoints willynilly until you have no idea what to use.

GraphQL is essentially the "Black" for Python Syntax but for Web APIs. Ever seen engineers fight over spaces vs tabs, 8 vs 4 spaces, whether a space before a colon? those fights happened a lot and then `black` came out and standardized it so there was nothing to fight over.

GraphqL makes things very clear and standard, but can't please everyone.

Unpopular opinion: I'm actually a fan of singe page specific endpoints. You get much easier debugging, easier to audit security, easier performance optimization an the imho pretty small price to pay is that it's "not elegant" and a bit of backend code
Yup. Especially when your api really only has 1 (possibly 2 in the case of a mobile app) real clients. People like to pretend their apis are generic but they aren’t. There’s a good argument to stop writing generic apis for your single application.
Typically the endpoints clients are multiple frontend developers. If the frontend is blocked for every page they need waiting on the backend to expose data that massively increases the costs of features and reduces the time for delivery.
This sounds like more of an org chart problem or a value chain management problem than a technical problem.
That's why we have Conway's law.
Frontend devs can write fake API calls which return the data they expect in the meantime.
This happens anyway.
Not nearly as often with Graphql and happens less and less as your backend and data models stabilizes.

Most of our frontend features now don't have backend changes and we were able to increase the ratio of frontend to backend devs.

This is the true purpose of REST. If you need to make multiple requests for a single operation you don't have enough endpoints.

The idea that resources and the underlying data needs to map 1-1 is wrong.

The modern web development practices are just insane.

The GP's idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.

On the other extreme, we have the HTTP 1.0 developers saying something like "networks are plenty of fast, we can waste a bit of it with legible protocols that are easier to make correct", while the HTTP 2.0 ones are all in "we must cram information into every single bit!"

Every place you look, things are completely bananas.

> idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.

For me, what's crazy is that there are "web" developers who can't just add the endpoint they need while working on a frontend feature, or "web" developers who can't just add an element or a page for testing the backend endpoint.

What ever happened to full-stack developers? The "frontend" and "backend" developer split is so incredibly inefficient that it's really jarring—you take something that should take 2 hours and magically, through tickets, delegation, and waiting for results (then repeat that for debugging, who knows how many times!), make it into a 2-3 day task.

I once reproduced (black-box style!) a two-week effort by a three-man team in six hours of work simply because I had PyCharm and IDEA opened side-by-side and could write code on both sides at the same time.

If someone has a good explanation for why the full-stacks that were once the norm went almost extinct, I'd be happy to give it a read!

For context: I work on large-scale browser apps that are closer in complexity to something like Linear or Obsidian than to your standard WordPress blog with some forms. E.g I'm currently working on a browser-based GIS tool for the financial sector.

I started my career as a full-stack developer, but went all-in on frontend because I felt I was spreading myself too thin. At one point I found that I could choose to be almost good enough at doing two different things or extremely good at one thing. I chose the latter option.

Modern browser apps are complex beasts, at least if you want to do them right. You obviously have to worry about all the technical bits --- HTML, CSS, JavaScript, your view library of choice, platform APIs like <canvas> and WebAudio, cross browser testing, bundle sizes, performance optimizations, techniques like optimistic rendering, all that good stuff.

On top of that, you also need to work closely with designers to make sure they know the features and limitations of the platform(s) they're designing for. More often than not, you end up being a sort of bridge between the backend devs, designers, and product managers.

A lot of times you end up doing design too, whether you like it or not. I've learned a lot about UI/UX design just because I often have to fill in the gaps where a designer forgot to include a certain error state, or didn't test their design on tablet screens, or didn't account for cases where a certain API might not be available.

I tried for many years to learn as much as I could about Django as well as React and friends. But it eventually got too much. I found that I wasn't able to keep up with both ecosystems, and I was producing code that wasn't very good. I could certainly build things quickly because I was familiar with all parts of the stack, but it came at the cost of code quality, security, stability, and robustness. I eventually decided to hang up my backend developer hat and focus exclusively on what goes on inside the browser (which can be a lot by itself these days!)

It's probably possible for a single individual to build a high-quality server-rendered MPA with some forms without making a mess of it. But that says more about how good Rails/Django/Laravel are than about the capabilities of any single individual. I don't think a single person could build a product like Linear end-to-end without cutting corners.

IMO the fact that being a full-stack dev is so taxing is an indication that the stack as a whole is just way too complex and overengineered. Which is rather obvious from looking at the state of affairs on the frontend side of things. Desktop GUI devs don't usually have those problems.
I don't think the claim is that a single developer should be able to build an entire enterprise product, but rather that a single developer should be able to implement the software side of a task end-to-end. The latter is a reasonable expectation when your software is monolithic.
> ... I don't think a single person could build a product like Linear end-to-end without cutting corners.

Cutting corners is a feature. I bet the Linear team is as pained as any, internally, at the tradeoffs they're making.

There is no way to know "what to get right" without going through it. So for 80% of the dev cycle the job is to cut corners to get to the 80/20, rinse and repeat forever.

This isn't against excellence and the dream of a beautiful product experience as your reply seems to convey.

If you absolutely need 2 people for that, they should be side by side.

But yes, that break-down of the problem is insane. People artificially split an atomic problem in two, and go create all of that extra craziness to try to solve the communication problem they made.

And then people go and push UX and UI tasks into the frontend developer, and ops tasks on the backend since them are there... What is insane again, because those are really incompatible tasks that can't be done with the same mindset.

And since it's standard, backend tools won't support the frontend and the other way around. So the insanity gets frozen on our tooling.

Because people have a limited appetite for complexity.

I wrote some front-end stuff back in the days but I've lost track of whatever is happening these days. jQuery to append some stuff took five minutes to learn, but learning react hooks takes a determined effort.

Likewise, adding a field to a graphql type is simple, but doing it with authorization, controlling n+1s, adding tests etc.. requires front-end folks to actually invest time in learning whatever back-end they're dealing with this time.

Everything is just a lot more complicated these days, and if you've been around for a while you may not be excited anymore by the churn, but rather fed up.

This is why the Rails community should be applauded in my book, for their dogged determination that we should keep it a “one person” framework. Yes it may not be as performant, type safe or flashy on the front end but my god it’s productive.

At my startup there are 7 devs who can all do tickets across the stack and as we grow I think it would be good if we could resist the pressure to silo and specialize

Many organizations would rather pay 3 people $120,000 each instead of paying 1 person $300,000 to do the same work, for a variety of reasons. Some good, some bad.
Microservice architectures and the associated explosion in complexity on both ends are to blame. When it takes twice the time to build something, it is natural to hire twice as many developers. Increased complexity drives specialization.
I think it's elegant. I loved MVVM in the postback world, and with SPAs, I see view-specific endpoints as the view-model continuing to reside on the server while the view was split off to the client-side.
I’ve heard this called the “backend-for-frontend” pattern.
Sounds good in theory. In practice, what I've seen is people don't just do single page endpoints with shared services, they follow this all the way down into the services and so you end up with lots of duplicate looking code. This isn't a problem until the application gets big enough and now you have business logic that should be the same spread out across an app. This leads to weird bugs where something that should work the same way doesn't depending on which page you're accessing things from. Of course, I don't a solution, solving things like this is what makes software engineering hard.
> In practice, what I've seen is people don't just do single page endpoints with shared services, they follow this all the way down into the services and so you end up with lots of duplicate looking code

There is no easy or simple or final solution. The solution is to do the hard work of software engineering and catch this stuff at code review time or to plan and do refactoring later when you realize something needs normalization. I wish developers would just accept that software engineering is hard and do the hard work necessary.

There's that whole "the best software engineer is lazy because he will automate stuff", but that does not give anyone a license to automate stuff in an unmaintainable way. Automating stuff is hard and there is no easy way out.

The backend for frontend pattern. Something most apps where frontend and backend are in the same organization (fullstack or otherwise) should have. It does wonder to maintainability and performance.

Even though we use GQL here, we still have a B4F, so it's browser -> B4F -> GQL -> Database.

The tooling we use make it trivial so it doesn't add any development overhead (often reduces it. No CORS bullshit for one), and our app goes ZOOOOOOM.

Came here for that. People in this thread basically reinventing GraphQL lol.

"We need a http API endpoint that gives you all the data for one page. And also be able to reuse parts of it for other pages." Yeah bro, this is GraphQL.

Agreed. Specific endpoints. I was on a project recently where a 40+ yr old dev was going crazy over "pure REST." It's embarrassing. Sure, have your pure REST but FFS create specific endpoints if you they make sense.
You are literally Transferring the REpresented State of a given page/view/screen.

Screen-specific REST endpoints will make their way as a default in to a JS-based framework in 2025 and people will pretend like this is some breakthrough advancement.

And finally HyperText will progress into the futuristic HyperCard
> With internal REST for companies I have seen so many single page specific endpoints. Gross.

Hardly gross. It is what it is and it’s universal across the domain. I bet Windows has internal APIs or even external ones that were created just for one page/widget/dialog of one app. It’s the nature of things at times.

It's analogous to a specific method in code. No idea why people go nuts over this.
I suspect that people don't think about it too hard and presume that they should have only one model - that is that their REST model should map directly to their DB model which should map directly to their internal domain model (the idea that there might be two-or-more-domains and not-all-of-them-are-public hasn't even yet crossed their minds).

When you realize that sometimes your REST model may be a view-model that maps to a different storage model optimized for online-transaction-processing, neither of which map to your internal domain model directly (there are adapters for that) _then_ you get somewhere ... but of course you then have to fight off the `transformToRestModel(transformToDomainModel(retrieveDbModel(theId)))` when the three models happily coincide for a time.

Some people treat codebases as an art project rather than engineering project.

I think this explains about three quarters of all engineering problems I've seen in corporate context, give or take.

A REST endpoint can be analogous to a specific method in code. just as much as a GraphQL field.

What people are excited about is that the frontend can request all the data it needs at once and the backend can efficiently serve it. Something not possible with REST without reimplementing something similar to GraphQL.

It's definitely possible, just not without tight coordination between frontend and backend. When your development teams are broken into silos with strong separation, this isn't feasible and GraphQL starts to make some sense. Otherwise, you just create an ad hoc endpoint that serves the exact data you need and call it a day.
Its gross because it is a waste.

An engineer had to spend time to make that specific API for that page instead of the frontend consumer using what was already defined and get all the resources with one call and 0 backend engineer needed for that new page.

On the other hand, it may take that engineer less time to create a page-specific endpoint than it would be to create something more generic that might serve other purposes (that may never come to pass), which may also involve talking to other teams, checking what future plans are, etc.

And that's assuming it's a new endpoint; if there's an existing endpoint that does almost what's necessary, they may need to check in with that team about what modifications to the endpoint would be acceptable, etc.

Single-page endpoints aren't great, but often times they're acceptable because they end up being a half-day task instead of a week-long slog.

That backend engineer wrote a single SQL query that joined some tables, ensured indices were used, and always executed in <1ms.

In graphql land you'd be doing multiple SQL queries, "joining" in the API layer, and spending 50ms per API call.

The entire point of graphql servers is that they're basically ORMs (or use an underlying ORM) that turn complex nesting into a single query's worth of joins. It won't beat hand-crafted sql from an expert, but if that's your preferred approach, debates on the relative merits of different query frameworks are all academic to you anyway.
I've never seen this kind of graphql server implementation that can automatically boil down a complex nested query to sensible SQL. It sounds like the best of both worlds. Do you have links?
What? GraphQL is purpose built to solve that in 1 Query. Not doing it in 1 query is on you not the protocol.

In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.

> What? GraphQL is purpose built to solve that in 1 Query. Not doing it in 1 query is on you not the protocol.

1 graphql query maybe. But that translated to a dozen SQL queries.

> In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.

The point you're missing is that for 1 graphql query the API did N+1 SQL queries, and then also joined them in JavaScript.

In the REST case the front end can switch to the efficient custom endpoint when it is implemented. In the graphql case it will never get any faster because the API has to stay generic.

A lot of graphql implementation end up moving the n+1 problem to the query resolver.
Is the flexibility worth the tradeoffs? Maybe in a company where you are adding new pages all the time with deeply nested relational data needs. But I would argue this is more rare than not. And I often find that frontend engineers aren’t as familiar with database queries and the load they are putting on the system with some of the graphql queries they are making. Flexibility for frontend has its own tradeoffs and I totally understand why a frontend engineer doesn’t want to have to wait for an endpoint to be finished. But this article outlines some of the issues you encounter later as you scale your team and system.

We use a schema first design where I am at and if a frontend person needs a new endpoint because the resource-only endpoints aren’t enough then they submit a pull request to the schema repo with a design for their endpoint they need. It gets approved and boilerplate is auto generated. Yes you have to wait longer, but 90% of the time (for our software) the resource endpoints work great.

Sorry what are the tradeoffs?

I'm not sure where this narrative comes from that GraphQL immediately means that the frontend time will have no idea what they are doing and will induce load on the system.

95% of my nested graphql fields are based on foreign key indexes so its almost no additional load to the system (only 1 query per "level" of GraphQL) to query the nested objects.

I restrict GraphQL to 5 levels and now I have 5 queries per API call instead of 5 Queries over 5 API calls.

The backend team exposes the fields that they know are efficient to query. They can restrict the depth of the GraphQL, number of fields, etc.

Every page is special. Using generalized API is waste of resources. Generalized APIs are hard to use and develop. It takes a genius to create a good generalized API (say SQL) and even those generalized APIs are not usable in web context (because of authorization and DDoS issues).

It's better to think about optimizing of creation specialized APIs.

This is like, an hour per endpoint. For maybe 30 endpoints (high figure for many apps) throughout the lifespan of your application. But let's say 3 hours: you're talking about 90 hours in total, over a period of 2 or more years. It's really not that much.

And GraphQL isn't free either; you need to actually implement that. It pervades your entire stack, too – it's not something you can just "put on top of it".

I think that in many cases, perhaps even most, GraphQL is a "spend 8 hours to automate a half hour task" kind of affair. Are there cases where that's not the case? Probably. Maybe your specific use case is one of them. But for the general case? I'm entirely unconvinced.

An experience engineer knows that a change that takes hour turns to days when a team is running at scale and needs to have CI approvals etc.

Why waste any time?

> And GraphQL isn't free either; you need to actually implement that

Rest isn't free. You have to actually implement also that and end up with a more limited API.

GraphQL libs in Python are equally as complex as the FastAPI etc..

And no one will be spending days working on it, they will work on other things while waiting for code reviews and such.

> GraphQL libs in Python are equally as complex as the FastAPI etc.

You need to account for the fact that anything can query anything, whereas a more traditional REST-type API will have much more limited codepaths.

This starts with just basic database maintenance and indexing, which in many case will be significantly more complex, to all sorts of other problems. This article already enumerates them so I'm not going to repeat them here.

You can't just handwave all of that away with "oh there's a library you can use".

If you think all of that is worth it then that's fine. As with many things it's a trade-off, and a bit of a judgement call too. But please, don't pretend this complexity doesn't exist, and that the trade-off doesn't exist.

If an app's API requires more than a couple of nested structures then, without knowing the specifics of the domain, I would venture a guess at a poorly defined data model is the issue vs. API technology.

For example, a lot of times people build out nice normalized table structures for online transactional apps. The UI/UX is pretty straight forward because end users typically only CRUD an object maybe a couple nested objects at a time. The API's are straight forward as well, likely following single responsibility principles, etc. Then comes along requirements to build UI's for analytics and/or reporting types of things where nearly the entire schema is needed depending on what the end user wants to do. Its the wrong data model for doing those types of things. What should be done is ETL the data from the OLTP schema into a data warehouse style schema where data is de-normalized so that you can build reporting, etc.

> Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.

With REST, though, that pain is visible to both sides. Frontend engineers generally don't want to make N+1 REST calls in a tight loop; it's a performance problem that they see and that is very visible in their Dev Tools. Backend engineers with good telemetry may not know why they get the bursts of N+1 calls that they see without asking the Frontend or digging into Frontend code, but they can still see the burstiness of the calls and have some idea that something could be optimized, that something is maybe too chatty.

There are multiple ways with REST to handle things: pagination, "transclusions", hyperlinks, and more. Certainly "single page endpoints" is a way as well, no matter how gross it is from REST theory, it's still a pragmatic solution for many in practice.

REST certainly can please everyone, given pragmatic compromises, even if it isn't very clear or standard.

> With internal REST for companies I have seen so many single page specific endpoints. Gross.

Single page endpoints is exactly what you want if you have more than 5 engineers in your company anyways.

It ensures that the endpoints are maintainable and future-proof when people are working on different features.

> It ensures that the endpoints are maintainable and future-proof when people are working on different features

How does GQL prohibit this? It encourages it by focusing on 1 stable API for everyone instead of a custom API endpoint for each case.

It doesn't. That's literally what GraphQL was designed for: To provide a resolver that sits between high latency clients and all of the various REST/RPC services in order to roll up all of the requests at datacenter speeds into one in order to overcome the latency issues.

But you still need all of the various single page REST/RPC endpoints to make use of GraphQL as it is intended. Some developers out there skip the REST/RPC part, making GraphQL their entire service, violating its principles.

If it works it works, but it does not come tradeoff free.

? why do you need a REST endpoint with GraphQL? Nearly every language has a GraphQL engine that integrated directly with the database and exposed functions as GQL directly without REST at all.
You don't, obviously – we already talked about how many don't. But if you don't have a graph of "microservices" in which to query, what do you need GraphQL for? Especially if all you are doing is returning the results from a database. The database will already have its own API. Why not just use it? The native API is always going be better than your attempt to translate.
I don't think it would hold true for very long, devs will have specific cases which will pile into the definitions.

That's why GraphQL examples usually focus on querying data from users and not how you are going to manage 10 different views of the same data.

> pile into the definitions

That is how REST works but is the opposite of the way GQL works.

You don't pile into existing defintions but extend the current definitions out. A team needing new data doesn't affect the consumption of other teams. which is not the case of REST where if one team needs to change a REST endpoint to return different shape of data, they have to verify the new shape with other teams.

With REST, the endpoints are team based if you have even a semi-competent CTO so you never have this problem, you just check who owns the controller and that's it.

With GQL though, good luck retracing who owns what field and what does it affect if you change the definition of what it returns, especially that you you are not even using it on the same language.

Bonus points here if you are managing something outside of your control like a mobile app.

One place I was at we used REST with a hydration service that ran as a proxy in front of the requests. That gave us most of the benefits of GraphQL and we only implemented it for the main resources (users, photos, and maybe one other thing). To minimize latency/faults the hydration service ran locally on every webapp/API server. I wasn't around for far too long after though to see how it turned out as it grew (if it did at all, the company sort-of went defunct--development-wise anyway).

I also recall, we had similar N+1 query problems in the REST API endpoints irrespective of hydrating the returned resources.

The biggest benefit of GraphQL I can see from a user perspective is that it lowers total latency especially on mobile with fewer round trips.

you don't think it's a benefit that you could get the benefits of a "hydration service that ran as a proxy in front of the requests" out of the box?

there's lots of other benefits for GQL: multiple queries per request, mutation/query separation, typed errors, subscriptions support.

I currently use GraphQL and have no problems with it specifically, I was merely sharing an experience using REST. Perhaps it adds a bit of latency/overhead due to the implementation/language that's used but with larger requests rounds downward.
The canonical REST solution of query params to add nested fields gets you quite far:

GET /myresource?extra=foo,bar

sure you over fetch a bit if you have multiple accessors.

But agreed, if you have highly nested data especially when accessed with multiple different query purposes then REST might not be the best fit.

I think GraphQL has been positioned as a general purpose tool and for that I am with the author, REST is a better go-to for most usecases.

I guess depending on the context of a simple app one level could be viewed as far.

Any more levels and you have now reinvented GraphQL

> Any more levels and you have now reinvented GraphQL

Sure, but reinventing the wheel can be good, particularly when the existing wheel technology is oblong, lumpy and constructed of stone and cheese.

I’m just bitter than GraphQL endpoints return 200 on errors. If you’re returning 200 on errors, then you’re not really doing HTTP; you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.

> you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.

Browser does not allow to access any ports with any protocols.

Often intermediate software gets in the way as well. For example with Kubernetes it's trivial to expose HTTP service using Ingress and it's not trivial to expose other protocols.

Another HTTP plus is that it's trivially secured by TLS and every developer knows API for that. Using TLS for your own protocol or QUIC is absolutely not trivial and probably more like arcane knowledge.

OData existed before GraphQL in the wild, it's possible to suggest GraphQL reinvented OData.

https://www.odata.org/

The point is GQL is currently the standard for querying dynamic nested data shapes not that it is the first or was the first.

Look at OData download stats on Pypi it had 2 downlaods the last day. Graphql-core for python? 624,201. that is not even on the same planet.

If you don't use GQL and want a system of querying nested data you will be using a less used protocol that is analogous to GQL so you might as well use the standard.

OData is still currently a standard for querying (among other things) nested data shapes. The point was it is a matter of fact that as a standard it predates GraphQL and as a matter of ecosystem perspective whether or not you think it is a more or less used protocol.

To trade anecdotes for anecdotes: On Nuget, the Microsoft.Data.OData package marked deprecated and marked with CVEs for critical bugs still has an average of 36.3K per day downloads. (Its successors combine for about double that.) The top GraphQL library (of the same name, unadorned) still only has 9.3K per day downloads. In .NET if you want a system of querying nested data (that is also "REST compatible"), why would you use a less used protocol like GraphQL?

> Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.

> With internal REST for companies I have seen so many single page specific endpoints. Gross.

As someone pointed out in reply to another comment, GraphQL is "a technological solution to an organizational problem." If that problem manifests as abuse of REST endpoints, you can disguise it with GraphQL, until one day you find out your API calls are slow for more obscure, harder to debug reasons.

> With internal REST for companies I have seen so many single page specific endpoints. Gross

That's an established pattern (backend for frontend). Like all patterns there are trade-offs and considerations to make, but it's certainly not a priori "gross".

https://learn.microsoft.com/en-us/azure/architecture/pattern...

Perhaps that's the case for server-side transactions and ORMs over REST and gRPC to coordinate rather than allowing the front-end to diddle with data however it likes.

And, resources with an index option obviously should have a db index or unique index.

The challenges with GraphQL are that it makes it too easy to DoS services, leak internal data, break referential integrity, and there were a great deal of tools, infrastructure, and monitoring systems already available for REST (and gRPC to a degree).

Company standards for REST and coding style can and should be set in the diff review pipeline. Another facet is setting standards to minimize duplication of effort or exposing a greater attack surface.

Harder to get a promotion when you’re doing something that’s old, boring, and just works.
The closer you are to the cutting edge, the more sliced up you’re gonna get.
Especially if your job title has "architect" in it.
XML still has its usage and is excellent to quickly validate documents, but JSON clearly won in this use case because it was kept simple, stupid.
I think IDLs and gRPC in particular are a meaningful advancement in the problem space. The ecosystem and productivity that they enable via programatic tooling were really eye-opening for me (The OpenAPI ecosystem is, imo, extremely poor). They also have better support for modern techniques like streaming and even load-balanced streaming.
Amen, brother! gRPC is the best thing since sliced bread. Typed APIs are just so nice to work with, and OpenAPI is very unsatisfactory.
I think us engineers have an inherent desire to try to innovate, and I think that is a good thing. Some problems will require a lot of wrong turns before finding the right path, but that is simply the nature of innovation
Agreed it's a good thing. Writing software today is still a slog that turns into a spaghetti bowl without focused intent by highly skilled devs. I think most devs realize this and really want there to be something better, and the only way to find something good is to find a lot of the bad around it first.

I like the thought experiment of adding a new persisted/editable field to an entity in a web app, if you've done full-stack you know all the layers that need to be touched to accomplish this and how lame most of the work turns out to be. After doing that 20 times while iterating, any dev worth their salt starts to wonder why it sucks so bad still and how it could be made better, and some will actually try.

JSON winning over XML is like saying CSV won over MySQL. They aren't equivalent.

Much like CSV, JSON isn't particularly standardised and different parsers and writers will do different things in some situations. Usually it doesn't matter, but when it does you're probably in for a lot of pain.

If you handle structured data and the structures might change over time, JSON isn't a good fit. Maybe you'll opt for JSON Schema, maybe that'll work for your use case, but with XML you can be quite sure it'll be reliable and well understood by generations of developers.

The tooling is generally very good, commonly you can just point your programming language to the XSD and suddenly you have statically typed classes to program against. Perhaps you'd like to store the data in a RDBMS? You can probably generate the DB schema from the XSD. If you want you can just throw JSON into MongoDB instead, but there will be very important tradeoffs. Same goes for UI, you can write some XSLT based on the XML schema and suddenly you get web views directly from API responses. Or you could use those classes you generated and have your GUI code consume such objects.

None of this is as easy with JSON as it is with XML, similar to how many things aren't as easy with CSV as with a RDBMS.

What's missing in ECMA-404? Never had a problem with JSON parsers or writers, using it all day every day for decades. It's crappy in some ways, sure, like lack of full floating point support, but standardization is not an issue.

XML is mostly already lost on the current generation of developers though, much less future developers. Protobuf and cousins generally do typed interchange more efficiently with less complexity.

It's focused almost entirely on syntax and ignores semantics. For example, for numbers, all it says is that they are base-10 decimal floating point, but says nothing about permissible ranges or precision. It does not tell you that, for example, passing 64-bit numbers in that manner is generally a bad idea because most parsers will treat them as IEEE doubles, so large values will lose precision. Ditto for any situation where you need the decimal fraction part to be precise (e.g. money).

RFC 8259 is marginally better in that it at least acknowledges these problems:

   This specification allows implementations to set limits on the range
   and precision of numbers accepted.  Since software that implements
   IEEE 754 binary64 (double precision) numbers [IEEE754] is generally
   available and widely used, good interoperability can be achieved by
   implementations that expect no more precision or range than these
   provide, in the sense that implementations will approximate JSON
   numbers within the expected precision.  A JSON number such as 1E400
   or 3.141592653589793238462643383279 may indicate potential
   interoperability problems, since it suggests that the software that
   created it expects receiving software to have greater capabilities
   for numeric magnitude and precision than is widely available.

   Note that when such software is used, numbers that are integers and
   are in the range [-(2**53)+1, (2**53)-1] are interoperable in the
   sense that implementations will agree exactly on their numeric
   values.
But note how this is still not actually guaranteeing anything. What it says is that implementations can set arbitrary limits on range and precision, and then points out that de facto this often means 64-bit floating point, so you should, at the very least, not assume anything better. But even if you only assume that, the spec doesn't promise interoperability.

In practice the only reliable way to handle any numbers in JSON is to use strings for them, because that way the parser will deliver them unchanged to the API client, which can then make informed (hopefully...) choices on how to parse them based on schema and other docs.

OTOH in XML without a schema everything is a string already, and in XML with a schema (which can be inline via xsi:type) you can describe valid numbers with considerable precision, e.g.: https://www.w3.org/TR/xmlschema-2/#decimal

GraphQL does define the size of its numeric types: Ints are 32 bits, Floats are 64, so if you have a bigint type in your db, you'd best be passing it around as a string. Any decent GQL implementation does at least check for 32-bit Int overflow. Several people have independently come up with Int53 types for GQL to use the full integer-safe range in JS, but the dance to make custom scalars usable on any given stack can be tricky.
There are a lot of proponents that some or all of the "JSON5" [1] improvements should be standardized by ECMA as well. Especially because there is a mish-mash of support for such things in some but not all parsers. (Writers are a different matter.) Primarily comments and trailing commas, are huge wish list items and the biggest reasons for all of the other "many" variant parsers (JSONC, etc).

[1] https://json5.org/

This is more of a concern for JSON configs and other such uses that are directly exposed to humans, but not really for machine-generated and machine-consumed data.
It differs from the RFC, notably "text" is valid JSON according to ECMA but not the RFC. I've come across JSON parsers stumbling on the bottom part of ASCII, for example. JSON -> internal representation -> JSON commonly leads to loss of information.

Sure, protobuf is nice, but more limited in scope and closer to a JSON alternative than an XML alternative.

I use JSON every other day and have been for decades.

Frankly, if a developer can't figure out XML then they aren't worth their salary. Age is no excuse here; as a developer your job involves figuring out how to work with technology you haven't used before.
XML parsers suffer the same fragmentation issues that JSON parsers do.

Go's XML parser straight-up emits broken XML when trying to output tags that have prefixed namespaces.

How come? Someone had a boring summer and made their own instead of using libxml2?
Its not JSON over XML. They're saying JSON REST won over _XML and SOAP_.
Those two are kinda orthogonal, and while there was some overlap for adoption, it was fairly common to serve XML over REST early on (because more languages and frameworks had proven-quality XML parsers out of the box, so it was easier for the clients to handle).

JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend. Then other languages caught up with their own implementations, although in some cases it took a while - e.g. for .NET you had to use third-party libraries until 2019.

> JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend

Browsers had XML parsers before they could handle JSON directly, and at the beginning there were complaints that JSON was harder to use for that reason. The reason why JSON won rapidly even for backend apps which never loaded it in JSON was ergonomics: every part of the XML world from the parsers to XPath/XSLT/XQuery to the rat’s nest of standards was plagued by the hairy-shirt “this is hard and should feel hard” attitude that has thankfully become less common. I saw so many people just burn out in the entire ecosystem because they got tired of unhelpful errors, pointless usability bugs around namespaces, low-quality or missing examples, and especially how common tools just stopped getting improved.

I maintain that the format would have been far more popular if all of the effort spent on standards work after the turn of the century had been suspended and the time directed to fixing things like the usability of namespaces in almost every parser, and hiring at least one person to work on libxml2 so developers could actually use features which shipped after 1999. Unfortunately it seemed like there were a ton of architects who really wanted to spend time building castles in the air and they just seemed to assume that someone else would do the boring parts of implementing it, but those people all jumped on JSON pretty quickly. I worked with a bunch of people who weren’t developers and the cycle of initial enthusiasm fading into “doesn’t this kind of suck?” with XML was depressing to watch having seen so much initial promise.

To claim that pretty much every .NET project adding Newtonsoft.JSON as a first step was somehow a problem is just strange. No adequate team would claim this to be a problem.
It was enough of a problem that Microsoft eventually saw it fit to come up with the official replacement.
It was made so that the ecosystem could continue to evolve, particularly in terms of performance and security hardening. But okay, what was the reason System.Text.Json introduced in your opinion? What were the egregious problems with Newtonsoft.Json?
> and most likely a lot of the software we write will cease to be running in a decade

If only that were true in my experience.