[EvanAnderson]

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice.

It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale").

There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a standard, all things considered. I still believe it was torpedoed because of lack of trendiness and "not invented here".

[0] https://www.w3.org/TR/webdatabase/

[daleharvey]

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs.

Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. (https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit...)

As a sidenote, I worked a lot with the WebSQL API and it was not a very good API in the slightest, immaturity may excuse some of its flaws, and it isnt like Safari did a much better job with IndexedDB, its just a buggy browser and thats where WebSQL was used most, but a large part of the problem is that it was bolting an API that assumed a single threaded client when that is not the reality with web pages where multiple tabs exist

[BulgarianIdiot]

> The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs.

Looks at Chrome

[da_chicken]

At the time, Internet Explorer was still fresh on everyone's mind. It's not like we haven't been exactly where we are now before.

[hitekker]

It’s funny that the top comment points out the obvious-- people want SQL in the browser-- and a NoSQL author immediately has to jump in to claim that SQL in the browser was wrong all along.

I think the NoSQL and MongoDB folks are worried for their careers.

[dmitriid]

> The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs.

And yet we're are now at a point where Chrome rams its own APIs through standards bodies, and there are no (and often won't be) any independent competing implementations.

[jeswin]

> And yet we're are now at a point where Chrome rams its own APIs through standards bodies, and there are no (and often won't be) any independent competing implementations.

Very few people are using Chrome-only APIs which are not in the standards yet. So it's not really a concern.

But otherwise, Chrome really has pushed the web forward more than any other browser. If it hadn't, native (and walled garden style) app stores would have totally taken over.

The web is in business (and thriving) as an application platform because it's being pushed forward relentlessly. The Storage Foundation API discussed in TFA is a good example.

[dmitriid]

> Very few people are using Chrome-only APIs which are not in the standards yet. So it's not really a concern.

Ah yes. But SQlite not having competing independent implementations somehow is?

Also, "not many people using something" is not as great an argument as you think it is. See, for example, the latest problem with browsers deciding to remove alert/prompt/confirm: https://dev.to/richharris/stay-alert-d

> The web is in business (and thriving) as an application platform because it's being pushed forward relentlessly.

A quote from the same article: "An ad company shouldn't have this much influence over something that belongs to all of us".

So somehow non-standards that Chrome pushes (many of which will never get a different implementation because both Safari and Mozilla consider them harmful) are good, and push the web forward. But SQlite is bad because there are no independent competing implementations.

Got it.

[scns]

From the first comment below the article:

> The Safari team did quite a lot of work re-styling their alerts and dialogs to be within the context of the webpage, yet phishing and scams that utilize this still run rampant on iOS. Repeated alerts are used to lock up the browser and make it unusable, forcing non-technical users to call a scam telephone number because they think their device was hacked.

A few bad actors ruining it even for totally different usecases.

[dmitriid]

> A few bad actors ruining it even for totally different usecases.

Same thing happened to Do Not Track... that was used for fingerprinting

[jeswin]

> Ah yes. But SQlite not having competing independent implementations somehow is?

Except that's not why it was culled. The reasons are discussed in great detail on this thread and elsewhere.

> non-standards that Chrome pushes (many of which will never get a different implementation because both Safari and Mozilla consider them harmful) are good

I'm saying nobody uses those Chrome only APIs, until they become standards (which requires acceptance by other vendors). Or are you against experimentation?

[dmitriid]

> I'm saying nobody uses those Chrome only APIs, until they become standards

Ah yes. Once Chrome releases something in stable, literally no one ever uses those APIs, no one. And even Google's propaganda machine doesn't tell you to use them (example: https://web.dev/usb/)

> Or are you against experimentation?

I'm not. What Google does isn't experimentation.

[SilverRed]

My feeling is that the web apis try to do too much. Why do we need a webSQL api. Why do we not just let websites create a file and then they can provide whatever kind of library they want. They could package a WASM version of sqlite and just work like they would as a desktop app.

That way you never have to deal with browser incompatibility or unchangeable specifications.

[TeMPOraL]

I'd agree with some of the existing web APIs. But I think some web SQL API should be built-in.

Why? Because it's a very common pattern. Almost fundamental. Take any project more complex than a toy calculator, and you'll quickly find places where the authors are hand-rolling relational operations.

Does your app have an array of records in memory, which it then searches for values, filters by conditions, and/or sorts? I'd give it a 50/50 chance that if you replaced that array with an in-memory SQLite table, and all operations on it with SQL queries on it, the result would be less code, more readable code and better performance[0].

It's not just the web - I'd argue that programming languages in general should all embed first-class in-memory relational database engines. It should be a part of the language standard - if it's easy to write this:

  struct Foo {
    int bar;
    date baz;
    string quux;
  };

  //somewhere in something
  Array<Foo> foos;

it should be easy to write this:

  table<Foo, index=[bar, quux]> foos;

(or some other, better way of describing intended data access patterns)

and get a thing that can be efficiently queried, filtered and transformed along dimensions specified, with the compiler turning your request into efficient bytecode/nativecode. Doesn't mean you have to write queries in vanilla SQL - there are more composable syntaxes out there. But arbitrary loops and map-reduces aren't that good either.

When you start looking at your data processing code as database operations, you'll see it everywhere. That map/reduce/zip blob? That's a JOIN query. That struct you're keeping in your event loop, that looks like:

  struct Schedule {
    Array<Task> tasks;
    PQueue<TaskId> toRun;
  };

that priority queue is an index on the tasks array, and looking at its front is just SELECT id FROM tasks ORDER BY priority LIMIT 1;. That Entity-Component-System pattern you're using in your videogame? That's literally a relational database. It was conceived of as such (and for massively multiplayer games, is often implemented as such). Wouldn't it be nice if the ability to express this came built into the language? With a column-oriented option for improved performance, too?

--

[0] - It's trivial to add indexes to SQL table. The engine will maintain them for you. Nobody habitually adds indexes to their own regular variables. They, along with the code to maintain them, would manifest as extreme code bloat. Meanwhile, in-memory SQLite is freakishly fast. If you're measuring performance in Python/Ruby units, you won't even notice the FFI overhead.

[afavour]

Seems like a great way to give every site a multiple MB dependency.

[SilverRed]

I think its fine for a web app to be multiple MB. Websites like HN/wikipedia/blogs have no need for an SQL database but if you are loading something like a full IM client, it makes sense to download a few MB to make it stable over all browsers.

[KronisLV]

Disagreed, this is a dangerous line of thinking that leads to using technologies like gwt, Vaadin, Blazor, Flutter or others that try to turn the browser into something that it's not suited to.

Sure, there could be cases where you have absolutely no alternatives to do something really specific, but in those cases I'd first invite you to reconsider whether what you're attempting to do actually needs to be a web app. Of course, people's thoughts will probably be split there.

However, the bottom line is still this - large sites load slower and cost more to load, they consume more battery and CPU, which on lower end devices could lead to system instability. If users don't outright leave your bloated solution (i.e. if they're forced to use it because of network effect) then in the case of any non-optional conditions (slow or unstable networks, low end devices, expensive data) their experience will be miserable.

Maybe have your IM client be a downloadable app instead? Better yet, why not have it be a native app instead of a bundled browser app, for excellent file size, small attack surface and great performance, while conforming to the OS look and feel?

Alternatively, why not make your IM client have a lightweight browser version that doesn't try to do everything under the sun? Personally, i feel like the answer is not to make browsers do more, but to try to do less yourself.

To that end, utilize server side rendering and pre-rendering with hydration when needed. Split some bundles and shake some trees. Compress as much as possible and use boring, common fonts while preferring local versions if available. Avoid videos in most cases and use smaller images, or vector graphics. The web can be a simple, beautiful and fast place if we make it one.

[pimterry]

> why not have it be a native app instead of a bundled browser app, for excellent file size

There are other valid arguments, but "use a native app for smaller file size" specifically doesn't make sense at all - the opposite is true.

While webapps may be larger than simple web pages, they're still far far far smaller than native apps, and using service workers etc, you do only pay the download size for a web app once, just like a native app.

For example: the Twitter Android app is a 21MB download. I just tested loading the Twitter PWA (https://mobile.twitter.com) logged in but with an cleared cache: it's a <3MB, including all the content visible in my feed on the first load.

The native app is _7+ times larger_.

I suspect the same or worse is true for the vast majority of other apps, e.g. Uber on Android is a 50MB download, while m.uber.com (reloaded after login with a cleared cache) gives you the same features with <1MB.

[talolard]

I subscribe to an alternative premise - "The browser is effectively an OS and thus a standardized distribution target". There are cases where the people I want to distribute to don't suffer from the constraints you mentioned. e.g. will tolerate a 20 second page load in exchange for the experience my product will provide.

Being able to reach all devices and have a super fast experience without consuming bandwidth is always good, but not always a high priority .

[TeMPOraL]

Funny, how this is the static/shared library problem again :). And where the hivemind seems to prefer static libraries for regular software, it's reverse on the web.

I suppose the problem is, on the web, you're redownloading the whole bundle each time you run. There are layers of caching in the browser that are supposed to help, but they're almost always defeated by a combination of app misconfiguration and developers' reluctance to use the cache given the ridiculous rate of redeployment of web software.

I'd still argue that SQL API is a prime candidate for a shared system-level library - AKA browser built-in. The concepts behind it have been worked out in the 1970s and honed through decades since. It's not experimental tech. The API won't be changing every month.

[hitekker]

I view the "standards" argument as a red herring for building a NoSQL db in the browser. Which, to this day, is slow, buggy and requires third party libraries to be usable [1]

For those who are able to stomach an uncomfortable political history instead of an easy, technical answer, you can take a look at [2]. It's interesting that 7 years later, many the folks who pushed hard to get rid of SQL in favor of NoSQL seem to no longer occupy positions of prominence in the industry.

[1] https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_A...

[2] https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...

[daleharvey]

I am familiar with Nolans article, I created PouchDB (the project he is discussing), you seem to have misred the post as it discusses the technical nuance and tradeoffs involved in the decision at many points entirely agreeing with the position against WebSQL. While Nolan came to the a different conclusion than I did (a point he made in the post) he laid out challenges very well and made it very clear there was no obvious technical answer.

Regardless of how you view it, the benefit of hindsight shows the exact thing that people warned would happen did in fact happen (a widespread venerability in SQLite exposed across various browsers). Its also a fairly strange point to be personally insulting people involved in the process whose careers are doing perfectly well.

[hitekker]

Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below.

Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.

[nolanl]

I wrote that article 7 years ago, and FWIW I would side more with Dale these days. It's probably a good thing we didn't just slap a half-baked API on top of SQLite and call it a web standard.

The biggest problem is that yeah, WebSQL tends to be faster than IndexedDB. Or at least it was back when I was working on PouchDB. Biggest issue IIRC was that joins were faster in SQLite than implementing the same thing in userland on top of IndexedDB. Browsers eventually shipped getAll/getAllKeys which also helped with cursor slowness.

I haven't looked much at the Storage Foundation API [1], but it seems like a more reasonable approach moving forward. Just give developers the low-level tools and let them build SQLite on top of it. Also the Chromium devs have been working on relaxed durability, which apparently improves IDB perf in some scenarios [2] (although still not as fast as Firefox it seems [3]).

[1]: https://github.com/WICG/storage-foundation-api-explainer

[2]: https://www.chromestatus.com/feature/5730701489995776

[3]: https://bugs.chromium.org/p/chromium/issues/detail?id=102545...

[dmitriid]

> Consensus & Standardization

> Firefox: Negative [1]

> Safari: Negative [2]

However, I fully expect Chrome to ship it in stable sometime soon, like they do with dozens of other APIs.

There are now four different half-baked storage/file api proposals, at least one of them is already in stable Chrome... it's a mess.

[1] https://github.com/mozilla/standards-positions/issues/481

[2] https://lists.webkit.org/pipermail/webkit-dev/2021-February/...

[tehbeard]

Having worked with IndexedDB quite a bit at my job, I can level three criticisms at IndexedDB.

1. The API is the dogshit hot mess you'd expect for a pre promise/async API.

2. The lack of partial/computed secondary indexes.

3. Apple/Safari does EVERYTHING in their power to break it. I refuse to believe it's incompetence at this point, it's actively malicious.

[andai]

PROLOGUE

Six houses, all alike in dignity, In fair IRC, where we lay our scene, From ancient grudge to new mutiny, Where civil blood makes civil hands unclean. From Oracle, that SQL seer of IndexedDB, To Google, the stronghold of search, We add Mozilla, the Web SQL killa, And Apple, peering from its mobile perch. Here, a storage war would set keys to clack, Tongues to wag, and specs to shatter, There was also Microsoft and Opera, Who don't really seem to matter.

THE PLAYERS

NIKUNJ MEHTA, of House ORACLE, an instigator JONAS SICKING, of House MOZILLA, an assassin MACIEJ STACHOWIAK, of House APPLE, a pugilist IAN FETTE, of House GOOGLE, a pleader CHARLES MCCATHIENEVILE, of House OPERA, a peacemaker

ACT 1

SCENE: A dark and gloomy day in Mountain View, or perhaps a bright and cheery one, depending on your IRC client's color scheme.

[sdiq]

Romeo and Juliet, for those that may not get it.

[dangerface]

Better to have something thats already considered standard than to invent a new one because you think you can do better than sqlite.

[matharmin]

SQLite as a library is amazing, SQLite as a spec for a standard web API is not.

The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web?

Alternatively you could start from the ground up and create a new database spec that is similar to SQLite, but more limited in scope. But then it will not be directly compatible with SQLite, and need a translation layer. The browser would likely have to re-implement large parts of SQLite for this to work. And you'd miss out on so much of the functionality that makes SQLite attractive to use.

Arguably that would still have been better than IndexedDB. All that IndexedDB has going for it, is that it is simple to implement. But even then Safari manages to get the implementation horribly broken so often.

What we need instead is to have a low-level storage api that can be used to implement other databases as libraries, including SQLite. The Storage Foundation API, mentioned in the post, might just give us that.

[treis]

>SQLite as a library is amazing, SQLite as a spec for a standard web API is not.

>The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web?

This seems like a case of perfect being the enemy of good.

[pornel]

I think the current state is fine. You ship your WASM-blob of SQLite, which has the exact bug-compatible version of SQLite that you've tested your app against. The browsers are not burdened with maintaining a huge API surface that can "break the web".

Otherwise you'd have to deal with different versions of SQLite in different browsers, most likely outdated, with many options turned off. SQLite is full of quirks and gotchas, so it's safest to ship your version in your app.

[devwastaken]

No, SQLite was not safe to be used with arbitrary queries. There were multiple memory vulnerabilities that allowed escape of the browser sandbox.

https://www.sqlite.org/cves.html

[SilverRed]

Would this not be solvable by just making it all arbitary WASM code? As long as the simple WASM sandbox is secure, it doesn't matter how buggy your internal SQLite implementation is.

[infogulch]

This article is about how to defend against potential websql vulns. https://www.sqlite.org/security.html

[nightpool]

This article only contains mostly information on adding more limits to avoid DoS—nothing there would harden WebSQL against SQLite vulnerabilities except for SQLITE_DBCONFIG_DEFENSIVE, which was only added in response to the zero-day bugs that were found in Chrome after implementing SQLite.

[SahAssar]

The justification was more that the standardization process requires at least 2 independent implementations, and nobody was working on rewriting SQLite from scratch.

I would have loved WebSQL, but it is reasonable to require multiple implementations for full standardization.

[mschuster91]

> and nobody was working on rewriting SQLite from scratch.

One could also have embedded a trimmed-down PostgreSQL or MariaDB into browsers.

[bzbarsky]

Those don't implement the same SQL as SQLite does, or as each other... What would the standard actually say?

[SahAssar]

I don't know the reasoning being not doing it, but I'd guess it's related to those DB engines not being geared or interested in embedded use-cases and that no SQL engine seems to entirely agree with any other how to handle certain parts of SQL.

[mschuster91]

Indeed, it would have been a lot of work trimming down any of the major FOSS RDBMS towards embedding them in a browser.

Regarding the SQL dialect - that could have been handled by specifying a standard SQL dialect that's then dynamically translated to the target engine's dialetc.

[fzzzy]

That sounds great! Let's do it.

[coliveira]

I don't buy this reasoning because they could have standardized only the interface, letting it open for developers to choose SQLite or some other future implementation. There is no need to standardize SQLite behavior, including bugs.

[aseipp]

What actually happens in that scenario is that one group now writes an adapter on top of SQLite to make it "standardized" to whatever interface you design, increasing the complexity and scope for bugs. To be clear: you're literally reimplementing something that SQLite already does, except probably much worse, in the name of the "standard."

Then that adapter is carbon copied by everyone, because again, nobody is going to reimplement an 80kLOC SQL database as well as whatever 10k LOC parser/lexer/typechecking adapter someone wrote, if they can avoid it. Then everyone just uses that forever, and you're back to square 1, using one implementation everywhere, which is the exact situation standards are supposed to avoid anyway.

The working group was correct to reject a "compromise" like that because that's never how it works out in practice; it's a submarine suggestion from the start. And a big part of this is all because, as evidenced by numerous responses in this thread, modern computer programmers seem to value their own immediate satisfaction and time over literally every other potential concern, no matter how significant.

[SahAssar]

That's not how the process works though. You could argue that they should change the process, but the reasoning behind it seems solid to me:

> Implementation experience is required to show that a specification is sufficiently clear, complete, and relevant to market needs, to ensure that independent interoperable implementations of each feature of the specification will be realized.

From https://www.w3.org/2015/Process-20150901/#implementation-exp...

[jlongster]

Yeah, it adds to the absurdity of all of this.

Although I do empathize with browers vendors. I worked at Mozilla at the time and was aware that this is a lot of things to think about when integrating something onto the web. I get why it happened, but practically speaking maybe it should have won. It's not like Chrome seems to care much about cross-browser standards these days.

I'm hopeful for a storage layer like this though: https://web.dev/storage-foundation/

It might actually be a better outcome if we get a storage layer with close to native performance, and then you can compile and db/lib/etc and it gets to use it.

[7952]

I think offset based file access could be really powerful just based on what people are achieving in the browser with things like Flat buffers, proto buffers and even http range queries.

[jmull]

Building off what others have pointed out about sqlite not being a good choice for a browser standard, I just want to note that the approach laid out in the article is exactly the right approach to integrate sqlite into a web app: a web app links a version from a well-maintained “distro”. The issues inherent in trying to use an implementation as a specification and standard go away, while a “distro” maintained by experts will lower the bar to entry.

I think devs should be happy, not sad. It looks like we’re finally getting it right, and will have a feasible way to add the incredible sqlite to the set tools we have available to make web apps.

Hopefully absurd-sql keeps going and browsers adopt a good storage standard.

[gunapologist99]

> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale").

This is almost certainly not even close to correct. There are substantial reasons why it wouldn't be a good idea, but this might be the biggest one: it's very hard to adequately sandbox an external C library.

(... and, also, Apple probably would prefer that the web didn't exist at all, but that's a different pandora's box...)

[EvanAnderson]

My "web scale" bit is snark. The "because it was SQL based" comes from the feeling I got at the time that Mozilla absolutely wouldn't accept anything was SQL-based. I strongly believe anti-SQL attitudes at Mozilla, and little else, are what killed WebSQL.

This is also how I justify my opinion that the "no independent implementations" was more of an afterthought excuse, and less of a primary motivation on Mozilla's part.

(I'm a little pressed on time here, but I could probably find more examples if I had more time to search... Sorry!)

Quoting Jonas Sicking from a W3C IRC log[0]:

> we've talked to a lot of developers

...

> the feedback we got is that we really don't want SQL

Quoting Maciej Stachowiak from the "[e-mail]" list[1]:

> Hixie has said before he's willing to fully spec the SQL dialect used by Web Database. But since Mozilla categorically refuses to implement the spec (apparently regardless of whether the SQL dialect is specified),

...

> At the face-to-face, Mozilla representatives said that most if not all of the developers they spoke to said they wanted "anything but SQL" in a storage solution.

The comments relating to JOIN in the comments on Mozilla's blog post comparing IndexedDB and WebSQL[2] betray the anti-SQL (and, arguably, anti-relational database) stance at Mozilla.

Mozilla's people didn't like SQL, so any excuse for dismissing WebSQL (or even a simplified SQL dialect that didn't have the "quirks" of SQLite) was a foregone conclusion.

And now here we are shipping >1MB transpiled WASM payloads around when we could have agreed on a query language feature set implemented in native code using a consistent back-end standard storage format.

[0] https://www.w3.org/2009/11/02-webapps-irc

[1] https://lists.w3.org/Archives/Public/public-webapps/2009OctD...

[2] https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-we...

[roca]

I have no doubt that Jonas told the truth that that's the feedback he got. I was never aware of Jonas or anyone else involved at Mozilla having some kind of "anti-SQL" prejudice.

WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that. It might even have required shipping a specific version of SQLite to make sure that query planning matches other browsers.

It seems that IndexedDB has turned out poorly for various reasons but one of those reasons is that Chrome has botched their implementation and that's not Mozilla's fault.

[TeMPOraL]

> WebDatabase, with or without a spec, boiled down to shipping SQLite. We didn't want the Web to depend on that.

Ironically, all the popular browsers already ship SQLite - they just not expose it directly. So, in a way, the Web already depends on SQLite. So does a lot of other technology - we're talking about one of the, if not the, most widely distributed pieces of software on the planet.

[roca]

That's true, but "in a way" does a lot of work there. "In a way" the Web already depends on the C++ standard library, but we aren't exposing that as a JS API.

[EvanAnderson]

I don't think Jonas was dishonest in any way. I think Mozilla talked to front-end Javascript devs who came from a "NoSQL" background (after all, Javascript devs are what begat "web scale" MongoDB) and who had disdain for SQL. I think that disdain sealed the fate of any SQL-based API in HTML5, irrespective of the involvement of SQLlite.

The "no independent implementations" was certainly a convenient way to dismiss away the already-implemented SQLlite-based solution. It's a valid argument for rejecting SQLlite as the spec, for sure. Implementations notwithstanding it seems like Mozilla was unwilling to even consider anything SQL-based.

The same tone comes out in the "Beyond HTML5: Database APIs and the Road to IndexedDB" Mozilla blog post[0]:

> In order to really get Web SQL Database right, we’d have to first start with defining a meaningful subset of SQL for web applications. Why define a whole other language, when more elegant solutions exist within JavaScript itself?

...

> We were resolved that using strings representing SQL commands lacked the elegance of a "web native" JavaScript API ...

That sounds like the same rhetoric used in the early MongoDB / NoSQL craze days to dismiss RDMS, ACID, JOINs, etc. Never mind that SQL is a battle-hardened DSL that has proven itself to be suitable to the task-- it wasn't "elegant" enough for the front-end devs Mozilla surveyed.

As an aside, I also find the quote from that post very amusing:

> In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter.

The example for a JOIN using IndexedDB in the referenced article[1] looks strained. It really doesn't look like it has "syntactic simplicity" compared to the WebSQL version above it.

[0] https://hacks.mozilla.org/2010/06/beyond-html5-database-apis...

[1] https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-we...

[kall]

The sad irony is that if HN is anything to go by, SQLite is super trendy now.

I feel like "just sqlite" was really a very practical idea and now, because it has already failed once, we can‘t really try it again, or can we?

Having the spec basically be "bundle one particular piece of software into the browser" might not be in the spirit of web standards, but on the other hand SQLite is so widely bundled into everything, does it really matter?

[EMM_386]

> The sad irony is that if HN is anything to go by, SQLite is super trendy now.

If you look at the recent StackOverflow survey, the majority of the developers only have around 5 years of professional experience in the industry.

SQL has been around since the 1970s and is still around in force for reason. There's a good chance a lot of developers, especially the enormous number on the front end, do not have experience with SQL or are just getting into back-end work where they are exposed to it.

There have been a lot of SQLite articles recently, as well as node.js libraries to query SQL. I have a hunch this may be why.

[kall]

Yeah I think so too. It‘s a cliche by now to say these things go in cycles but they do. If we like this part of the cycle, let‘s ride it and let SQL get hyped, even though its been around forever and never went away.

[411111111111111]

What gives you the confidence that it's going on circles? I cannot see our industry repeating the mantra that NoSQL solves all our persistence problems ever again. There are use cases for them, but not everything is a good fit and I'm pretty sure we found that out as an industry.

I'm sure there are always going to be individuals that will claim that they're better at everything, but that doesn't really mean anything. There are people that believe in a flat earth as well after all

[coliveira]

The cycles don't happen exactly. But I'm pretty sure they will come up with something new that in the end is the same as NoSQL and give it a trendy name, so young developers will believe they found gold and run with it... It is just the way these things keep happening.

[regularfry]

The cycles are longer than you're imagining. Relational database folk have been railing against the incursions of one generation of non-relational database type or another since at least the 1980's.

[smackeyacky]

Actually, relational databases are the relative newcomers. The limitations of things like CODASYL were well known to 1960s programmers and Codd's relational model sought to address them.

Why the non-relational databases keep getting reinvented has been a bit of a mystery when there is already a rich history of development to look to. I get the feeling a lot of the industry isn't much into history, especially of pre-micro computer systems.

Network databases, key-value stores, graph databases, commercial offerings like Pick... you'd think the NoSQL people would have looked into it all before proclaiming their new found solutions, but apparently not.

[scns]

Interesting, you wrote circles but OP wrote cycles, which can be circular, sinuslike, shaped like a sawtooth/ramp-wave (LISPs AI-winter anyone?) or even deformed variations with a bit of randomness or reinforcing patterns in them. Hypecycles and speculation bubbles come to mind.

This reminded me of the writings of a german Philsopher, Heidegger maybe? The proposition was, that development happens in cycles similar to a pendulum swinging while moving upwards. Thesis and Antithesis leading to Synthesis.

[mr_toad]

> the mantra that NoSQL solves all our persistence problems ever again.

That argument was more usually a straw man than a good faith belief.

On the other hand the belief that SQL will solve all our problems seems to be legitimately held by some people.

[jkcxn]

If anyone else is curious https://insights.stackoverflow.com/survey/2021#developer-pro... Most people responding have 5 or more years of experience

[endisneigh]

You’re both right - most people have coded for more than 5, but most people have been coding professionally for 5 or less.

[jkcxn]

No I'm talking about professionally, 64.33% of respondents have 5 years or more professional experience

[otabdeveloper4]

There's no such thing as "the SQLite" that you could even theoretically bundle.

Which version do you want to bundle? With which compile flags and which extensions? Do you have a checklist for bug-for-bug compatibility? Because you'll definitely need one when a future SQLite releases a security patch and breaks a million webpages.

[TehShrike]

I used it back in the day and was quite sad when it disappeared. Nolan Lawson does a good job telling the sordid tale at https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...

[mickael-kerjean]

same here. Firefox pulling out from websql is the reason I stopped from using it and recommending it. If the standard wasn't good enough, it should have been improved and refined, not thrown out.

[zzzeek]

first thought, was there a "standard" for key/value stores? mongodb the reference implementation?

overall, if you looked at HN like five years ago, every DB headline was key/value, mongodb, maybe some cassandra / couchdb, links to the "web scale" cartoon.

these days, it's SQL SQL SQL, with a heavy dose of SQLite and PostgreSQL. SQL survived the key/value fad despite the nebulousness of a workable "standard" (yes there's a SQL standard but no vendor DB implements all of it or doesn't add many many features, syntaxes, and behaviors on top of it). In particular SQLite recently seems to look to Postgresql for guidance on new syntaxes such as how it implemented upsert, it's RETURNING syntax is explicitly from PostgreSQL, and it interestingly uses the same "VACUUM" term for db utility cleanup.

[apavlo]

> SQL survived the key/value fad

SQL has survived every fad since the 1970s:

Stonebraker "What Goes Around Comes Around"

https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/pape...

[jbergens]

I think sql is a bad fit for front end code in js. We may have ended up with another fight over ORMs. The Vietnam war of computer science as I think it was called.

[chaps]

I couldn't disagree any more, at least in my specific field, where my data is generally static and extremely relational. Writing JS to do what SQL does is... a pain. But having SQL on both the frontend and backend would dramatically simplify the translation abstractions between the two - for a similar reason that node on the backend simplifies translation abstractions. It ultimately means less code and less thinking.

[SilverRed]

How else would you suggest full applications run in the browser. Desktop apps make use of SQLite all the time even when they are mostly server backed.

An example is that Telegram maintains a local db which stores the most recently accessed content so you can scroll through past messages and media without having to request them every time you view them.

[lmm]

Desktop apps use SQLite because it's the embedded structured datastore that already exists, not because they want to use SQL. If it was easy to embed, IDK, some kind of file-backed redis, I'm pretty sure a lot of apps would use that instead.

[qwerty456127]

> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale").

Whenever I need simple (but indexed) key-value (unless that's a hi-load server-side) I always just use SQLite anyway. I really don't understand why do we need any data storage other than SQLite (and HDF5 perhaps) to exist on the client side.

[kwhitefoot]

> not trendy "key value"

You can put anything in an SQLite record so it can certainly be used as a key-value table. Where at least the values can be arbitrary binary blobs.

[niutech]

You can still use WebSQL in Chrome & Safari and use a polyfill backed by IndexedDB for Firefox using YDN-DB: https://yathit.github.io/ydn-db/doc/setup/polyfill.html

[gunapologist99]

Another, probably not insurmountable, issue with SQLite in the browser (without having a formal specification that could produce cross-language alternatives):

Using SQLite results in a new, built-in reliance on a C library. Even though most languages can use C library bindings, it does present some issues in some build scenarios, such as static builds, and, given the enormous scope of SQLite, would prevent anyone from ever achieving a 100% rust browser.

[Cyberdog]

I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for?

At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime.

[lmm]

> I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for?

A 100% memory-safe browser is definitely something I'm clamouring for.

> At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime.

Mozilla is working on the rendering engine. Memory-safe JS runtimes already exist (e.g. Nashorn).

[bayesian_horse]

I think security had more to do with it...