[cryptonector]

The problem with RESTful requiring hypermedia is that if you want to automate use of the API then you need to... define something like a schema -- a commitment to having specific links so that you don't have to scrape or require a human to automate use of such an API. Hypermedia is completely self-describing when you have human users involved but not when you don't have human users involved. If you insist on HATEOAS as the substrate of the API then you need to give us a schema language that we can use to automate things. Then you can have your hypermedia as part of the UI and the API.

The alternative is to have hypermedia for the UI on the one hand, and separately JSON/whatever for the API on the other. But now you have all this code duplication. You can cure that code duplication by just using the API from JavaScript on the user-agent to render the UI from data, and now you're essentially using something like a schema but with hand-compiled codecs to render the UI from data.

Even if you go with hypermedia, using that as your API is terribly inefficient in terms of bandwidth for bulk data, so devs invariably don't use HTML or XML or any hypermedia for bulk data. If you have a schema then you could "compress" (dehydrate) that data using something not too unlike FastInfoSet by essentially throwing away most of the hypermedia, and you can re-hydrate the hypermedia where you need it.

So I think GP is not too far off. If we defined schemas for "pages" and used codecs generated or interpreted from those schemas then we could get something close to ideal:

  - compression (though the
    data might still be highly
    compressible with zlib/
    zstd/brotli/whatever,
    naturally)

  - hypermedia

  - structured data with
    programmatic access methods
    (think XPath, JSONPath, etc.)

The cost of this is: a) having to define a schema for every page, b) the user-agent having to GET the schema in order to "hydrate" or interpret the data. (a) is not a new cost, though a schema language understood by the user-agent is required, so we'd have to define such a language and start using it -- (a) is a migration cost. (b) is just part of implementing in the user-agent.

This is not really all that crazy. After all XML namespaces and Schemas are already only referenced in each document, not in-lined.

The insistence on purity (HTML, XHTML, XML) is not winning. Falling back on dehydration/hydration might be your best bet if you insist.

Me, I'm pragmatic. I don't mind the hydration codec being written in JS and SPAs. I mean, I agree that it would be better if we didn't need that -- after all I use NoScript still, every day. But in the absence of a suitable schema language I don't really see how to avoid JS and SPAs. Users want speed and responsiveness, and devs want structured data instead of hypermedia -- they want structure, which hypermedia doesn't really give them.

But I'd be ecstatic if we had such a schema language and lost all that JS. Then we could still have JS-less pages that are effectively SPAs if the pages wanted to incorporate re-hydrated content sent in response to a button that did a GET, say.

[recursivedoubts]

HATEOAS is for humans:

https://intercoolerjs.org/2016/05/08/hatoeas-is-for-humans.h...

[cryptonector]

SPAs are for humans, but they let you have structured data.

That's the problem here. People need APIs, which means not-for-humans, and so to find an efficient way to get "pages" for humans and APIs for not-humans they invented SPAs that transfer data in not-for-humans encodings and generate or render it from/to UIs for humans. And then the intransigent HATEOAS boosters come and tell you "that's not RESTful!!" "you're misusing the term!!", etc.

Look at your response to my thoughtful comment: it's just a dismissive one-liner that helps no one and which implicitly says "though shall have an end-point that deals in HTML and another that deals in JSON, and though shall have to duplicate effort". It comes across as flippant -- as literally flipping the finger[0].

No wonder the devs ignore all this HATEOAS and REST purity.

[0] There's no etymological link between "flippant" and "flipping the finger", but the meanings are similar enough.

[recursivedoubts]

Yeah, that was too short a response, sorry I was bouncing around a lot in the thread.

The essay I linked to somewhat agrees w/your general point, which is that hypermedia is (mostly) wasted on automated consumers of REST (in the original sense) APIs.

I don't think it's a bad thing to split your hypermedia API and your JSON API:

https://htmx.org/essays/splitting-your-apis/

(NB, some people recommend even splitting your JSON-for-app & JSON-for-integration APIs: https://max.engineer/server-informed-ui)

I also don't think it's hard to avoid duplicating your effort, assuming you have a decent model layer:

https://htmx.org/essays/mvc/

As far as efficiency goes, HTML is typically within spitting distance of JSON particularly if you have compression enabled:

https://github.com/1cg/html-json-size-comparison

And is also may be more efficient to generate because it isn't using reflection:

https://github.com/1cg/html-json-speed-comparison

(Those costs will typically be dwarfed by data store access anyway)

So, all in all, I kind of agree with you on the pointlessness of REST purity when it comes to general purpose APIs, but disagree in that I think you can profitably split your application API (hypermedia) from your automation API (JSON) and get the best of both worlds, and not duplicate code too much if you have a proper model layer.

Hope that's more useful.

[cryptonector]

Thanks, I appreciate the detailed response.

> So, all in all, I kind of agree with you on the pointlessness of REST purity when it comes to general purpose APIs, but disagree in that I think you can profitably split your application API (hypermedia) from your automation API (JSON) and get the best of both worlds, and not duplicate code too much if you have a proper model layer.

I've yet to see what I proposed, so I've no idea how it would work out. Given the current state of the world I think devs will continue to write JS-dependent SPAs that use JSON APIs. Grandstanding about the meaning of REST is not going to change that.

[recursivedoubts]

I've built apps w/ hypermedia APIs & JSON APIs for automation, which is great because the JSON API can stay stable and not get dragged around by changes in your application.

As far as the future, we'll see. htmx (and other hypermedia-oriented libraries, like unpoly, hotwire, data-star, etc) is getting some traction, but I think you are probably correct that fixed-format JSON APIs talking to react front-ends is going to be the most common approach for the foreseeable future.

[cryptonector]

If you want JS-lesness and HATEOASnes then maybe if we had an automatic way to go from structured data to HTML... :)

[eadmund]

> I think you can profitably split your application API (hypermedia) from your automation API (JSON)

Why split them? Just support multiple representations: HTML and JSON (and perhaps other, saner representations than JSON …) and just let content negotiation sort it all out.

[recursivedoubts]

I tend not to like content negotiation:

https://htmx.org/essays/why-tend-not-to-use-content-negotiat...

[deniz-a]

The "structured data with programmatic access methods" sounds a lot like microformats2 (https://microformats.org/wiki/microformats2), which is being used quite successfully in the IndieWeb community to drive machine interactions with human websites.

[cryptonector]

Thanks for the link!

[riwsky]

> The problem with RESTful requiring hypermedia is that if you want to automate use of the API then you need to... define something like a schema -- a commitment to having specific links so that you don't have to scrape or require a human to automate use of such an API

We already have the schema language; it’s HTML. Stylesheets and favicons are two examples of specific links that are automatically interpreted by user-agents. Applications are free to use their own link rels. If your point is that changing the name of those rels could break automation that used them, in a way that wouldn’t break humans…then the same is true of JSON APIs as well.

Like, the flaws you point out are legit—but they are properties of how devs are ab/using HTML, not the technology itself.

[cryptonector]

HTML is most decidedly not a schema language.

[alexpetros]

> The alternative is to have hypermedia for the UI on the one hand, and separately JSON/whatever for the API on the other. But now you have all this code duplication.

What code duplication? If both these APIs use the same data fetching layer, there's no code duplication; if they don't, then it's because the JSON API and the Hypermedia UI have different requirements, and can be more efficiently implemented if they don't reuse each other's querying logic (usually the case).

What you want is some universal way to write them both, and my general stance is that usually they have different requirements, and you'll end up writing so much on top of that universal layer that you might as well have just skipped it in the first place.

[cryptonector]

I've worked with a HATEOAS database application written in Ruby using Rails. It still managed to have HTML- and JSON-specific code. Its web UI was OK, but not as responsive as a SPA.