[lostinthefield]

Next.js (and probably its competitors) does this same thing (https://nextjs.org/docs/basic-features/data-fetching#getstat...), just much more elegantly and modularly, allowing you to work with any headless CMS while still maintaining the benefits of a static, "baked" dataset that gets deployed alongside static code. It just caches API responses at build time into local JSON.

The thing is, no matter how you bake the data, it has to be edited somewhere. That "somewhere" is rarely good as Markdown files or or SQLite for all but the most trivial implementations. In real-world mid-sized websites, content editors want something easier to work with. Having an actual CMS, even if it's just a decapitated Wordpress, makes their lives much easier. And you can still get the benefits of baked data.

With incremental static regeneration (https://www.smashingmagazine.com/2021/04/incremental-static-...), you also get the benefit of post-build on the fly revalidation and "rebaking" without requiring a full rebuild, courtesy of a server that checks for data updates in the background and rebuilds only the affected pages.

[simonw]

There's nothing to stop you implementing a WordPress-style CMS as part of this pattern, then baking its database on every deploy and shipping that as a copy.

It means users won't get instant deployment of their edits though, which depending on your use-case may be a non-starter.

The Mozilla.org site I reference in the article appears to use Django (potentially with the Django Admin CMS tool) to manage the data, but then bakes and distributes a SQLite file with the data that has been managed by Django.

[lostinthefield]

> It means users won't get instant deployment of their edits though, which depending on your use-case may be a non-starter.

That's what the "incremental static regeneration" I mentioned does. Every time the data changes, Next.js catches it within your specified revalidation period, and then rebakes only the affected pages by updating their local caches. It's magical.

Better still, with this system, you never have to manage the database layer yourself. There is a CMS, yes, but if you go headless in the cloud, that's someone else's scaling problem. Any edits, whether at build time or subsequently, get baked into static HTML + JS that can be served from any CDN.

[simonw]

I think I see what you mean - so your production site is hosted entirely from a static CDN, but you have an intelligent build process running somewhere that can re-generate and then re-publish just a subset of the files when the underlying data changes?

[leerob]

Correct. More information here https://www.smashingmagazine.com/2021/04/incremental-static-...

[lostinthefield]

Yes, exactly.

The downside is that most of that is black-box magic. It's not clear to me whether it's a secret npm server, some serverless function, or something else entirely doing that "intelligent build process". But by and large it does work, way better than I expected it to.

In detail:

1) On static pages, you can set a revalidation period (say 60s)

2) Those static pages are always served from the CDN cache, regardless of whether there is a set revalidation. Visitors always get baked HTML + cached JSON data, regardless.

3) Visitors 1, 2, 3, 4 visit within moments of each other, and each see the same cached page.

4) Visitor 5 happens after the 60s revalidation period. They STILL see the same cached pages as visitors 1-4, BUT this also triggers a staleness check against the data origin. (This is the part that's black magic; I'm not really sure how it's doing that. It "just works" if hosted on Vercel; on other providers it may be necessary to spin up a separate `next start` node server. It's entirely unclear to me).

5) Behind the scenes, Next.js via its magic sees new data and rebuilds the affected page. It takes a while, though (maybe a minute or so depending).

6) Visitor 6 visits at 65 seconds, not quite enough time for the new page to have been built. They still see the same cached version as visitors 1-5.

7) But visitor 7 visits a minute later, at 150 seconds, and by this time Next has updated the cache. Visitor 7+ see the newly baked page, with the updated data.

8) The cycle repeats every 60 seconds.

So in production, new visitors will see updated baked pages pretty soon after the data change is made in the CMS. Caveats: it's not "instant" per se (just soon), and it requires a sacrificial visitor (visitor 5 in our example) -- or a editor or bot pretending to be a visitor -- to trigger that staleness recheck by visiting the page.

If you need truly realtime, pushed updates straight from the CMS into your Jamstack, that requires even more workarounds on top of the Next.js ISR that I described above. Some CMS vendors have proprietary solutions to this, like DatoCMS's real-time updates API (https://www.datocms.com/docs/real-time-updates-api) but I don't think there is necessarily a solved, best-practices model to refer to. Basically some sort of pub/sub using workers and server-sent events. But that gets pretty complicated vs clientside polling of updates. A slightly different problem from updating baked data in near- (but not actual) real-time.