[avl999]

I wonder how an alternate timeline might have played out if Richard Hipp had not named it "SQLite" and instead called it "SQLightning" or "SQLExpress" or something like that. For much of its lifetime, SQLite wasn't taken seriously despite being an extraordinary technology and part of me is convinced it was in large part due to "Lite" being in the name.

[dimitrios1]

SQLite has been taken seriously by serious people for decades. We just don't hear about it in the cool, hipster startup circles because it isn't sexy. But it's steadily grown to be the most deployed embedded database in the world, in millions of devices, and have specialized strands that went through military and medical grade formal validations to end up in airplanes, fighter pilots, naval ships, medical devices.

You have to look where SQLite isn't rather than where it is these days.

[tptacek]

The shift isn't in SQLite being taken seriously behind the scenes, or as a better file format. It's in SQLite being taken seriously as an alternative to n-tier database in full stack applications.

[rrdharan]

I don’t necessarily disagree but I still get an Upton Sinclair vibe from fly.io folks declaring that this shift is happening as a matter of fact / by fiat.

[tptacek]

Why? We don't have an SQLite offering; we do Postgres.

[sgbeal]

> ... in millions of devices...

cough _Billions_ cough with a capital "B". Very nearly every non-trivial electronic device built over the past 10-15 years. (Non-trivial being very roughly: "anything with a UI or having the potential to run one.")

[simlevesque]

You may already know but it's SQL-ite (sequel-ite), like graphite, dynamite, sulfite.

https://en.wiktionary.org/wiki/-ite

[srcreigh]

Not even Sequel-ite. But S.Q.L.-ite. That's how Dr Hipp says it, anyways.

[sgbeal]

FWIW... i've been in Richard's presence several times when he was asked how to pronounce it by various people and he's invariably answered, "pronounce it however you like!" (i belong to the ess-queue-lite school of thought.)

[dec0dedab0de]

I love when projects have cute puns in the name. I always am sure to correct people so they don't miss out on the fun. Luckily most developers are pedantic and like language enough that they don't get too annoyed with me. Same thing applies to PostgreSQL, it drives me nuts that sqlalchemy is not sqlchemy.

completely off topic at this point, but Jim Cornette reading paid advertisements for codecademy was very funny during the pandemic. He insisted on pronouncing it code-cademy while his co-host tried to correct him.

[avgcorrection]

Such ambigious names remind of “Rust”. It’s such an um-actually name (um actually it’s not first and foremost about oxidization it’s about fungi…)

[randomdata]

> it’s not first and foremost about oxidization it’s about fungi…

Um, actually this[1] suggests that it originally wasn't about anything other than seeming like a good name and that any deeper explanation that has been given was made up. Of course, by the same token, "it seemed like a good name" could have also been made up. But it appears that the team eventually embraced it in the oxidization[2] sense.

[1] https://www.reddit.com/r/rust/comments/27jvdt/internet_archa...

[2] https://bugzilla.mozilla.org/show_bug.cgi?id=680521

[avgcorrection]

Um actually I believe more in community consensus with regards to naming provenance as opposed to authorial intent

ad nauseam

[randomdata]

Fair, but community consensus is that of oxidization to reflect how the Rust community comes in, takes over, and ruins every discussion about programming by finding some way to mention Rust. It is very much a virtual oxidization process.

Case in point: This discussion was originally about SQLite.

[EGreg]

I had no idea! I was sure it was “SQL Lite” ever since I heard about it a decade ago. Its support for mostly text strings while MySQL and Postgres supported a myriad types made me think it’s just a small embedded “lite” library for when you wanted to bundle a mostly SQL database with your app lol

[geewee]

Huh, I had no idea!

[tveita]

I've used SQLite for toy services "in production", and it was really just as bad as people think it is. Sure, you could handle a large amount of read-only queries, but it only took a tiny bit of write traffic in the mix to make the random latency spikes jarring.

This was pre-WAL, presumably enabling WAL would help a lot (but is still not the default, so beware). But the caveats were real, it's not like people just took one look at the name and though "'SQLite?' I better put a big warning in our documentation to not use this in production."

[LAC-Tech]

This was pre-WAL, presumably enabling WAL would help a lot

Indeed it would!

"WAL provides more concurrency as readers do not block writers and a writer does not block readers. Reading and writing can proceed concurrently."

https://www.sqlite.org/wal.html

I think the people advocating for SQLite to be used in more places are all assuming write ahead logging is enabled.

[chasil]

There are many problems with WAL, as acknowledged by Dr. Hipp in the recent olap/duckdb paper.

The chief problem that I see with WAL is that it breaks ACID with databases that are ATTACHed, as the documentation shows:

https://sqlite.org/lang_attach.html

[simonw]

What are the other problems?

[chasil]

OK, I'll download the PDF onto my phone and get the quote...

...I forgot how significant these problems are. These are quite serious.

"However, WAL mode has notable disadvantages. To accelerate searching the WAL, SQLite creates a WAL index in shared memory. This improves the performance of read transactions, but the use of shared memory requires that all readers must be on the same machine. Thus, WAL mode does not work on a network filesystem. It is not possible to change the page size after entering WAL mode. In addition, WAL mode comes with the added complexity of checkpoint operations and additional files to store the WAL and the WAL index."

https://vldb.org/pvldb/volumes/15/paper/SQLite%3A%20Past%2C%...

https://news.ycombinator.com/item?id=32684424

[yread]

It also complicates backups. I have another process that .backups the database but even when opening it in read-only it creates a .wal file. My first version of the backup script didn't delete it afterwards and the normal process didn't have the right to overwrite the backup's wal file so opening the db failed

[gigatexal]

How often is attach really used though?

[chasil]

How often are distributed transactions used elsewhere with two-phase commit?

All the time. I suspect this is similar.

[gigatexal]

Right but that’s bending SQLite a lot no? I mean it’s meant to be used as a file that has a sql interface. But I digress.

[simonw]

WAL was added 2010-07-21, so pre-WAL is over 12 years ago now!

[hinkley]

Lots of changes with branch prediction in that time too.

[tptacek]

I don't think it has much to do with the name. I think the biggest perceptual impact on SQLite came from Rails, and its default for using SQLite as a test database while strongly discouraging people from using it in production.

[post-it]

Django too.

> By default, the configuration uses SQLite. If you’re new to databases, or you’re just interested in trying Django, this is the easiest choice. SQLite is included in Python, so you won’t need to install anything else to support your database. When starting your first real project, however, you may want to use a more scalable database like PostgreSQL, to avoid database-switching headaches down the road. [0]

[0] https://docs.djangoproject.com/en/4.1/intro/tutorial02/

[tptacek]

Sure, Django too.

[0x457]

To be fair, serving any kind of traffic with Rails required multiple replicas of Rails application running. So SQLite wasn't an option for production for any kind of production rails workload.

Then most hosting for rails were stateless, so you had no way of storing SQLite on disk.

And finally, for serious production you need high availability and SQLite couldn't offer that.

[Abishek_Muthian]

Would Go's concurrency be an issue for SQLite too?

Edit: I looked into a common Go driver for SQLite[1] and the FAQ reads,

> Can I use this in multiple routines concurrently?

> Yes for readonly. But not for writable. See #50, #51, #209, #274.

Every time I see a blog post from fly.io reg SQLite I'm tempted to use it for my next project, But the need to rewrite my framework for limited data types and the doubts regarding concurrency keeps me away.

[1] https://github.com/mattn/go-sqlite3

[tptacek]

My general M.O. for SQLite in Go is to have a single writer thread with a channel of closures to apply transactions, and a shared db handle for reads. Works fine.

[Abishek_Muthian]

That's an useful information, Thanks for sharing.

[randomdata]

I think it's older than that, even predating SQLite. Back when Perl/CGI was all the rage it was common to store data in files. But the web was growing fast and scaling files was hard. Pushing that workload onto the heavyweight RDMBS was seen as the solution, reasonably so in some cases, but that also brought the cargo culting along with it.

When SQLite arrived it got lumped in with file storage that can't scale. Rails only perpetuated what everyone was already thinking.

[nigma]

There are many reasons for using other databases like PostgreSQL or MySQL in production - feature set, (managed) hosting, backup and replication, etc.

SQLite is primarily embedded/local database and cannot be easily separated and shared over network [1] between multiple disposable backend/worker instances.

[1] https://12factor.net/backing-services

[tptacek]

Yes, that's what Rails (and Django) say about it.

[deepstack]

yeah when it comes to embbed, if one is willing to go the java road, then H2 is an excellent DB to use. It is some what Postgres compliant.

[d23]

Interesting. I didn't realize it wasn't for "lite".

A friend of mine made a similar point about GIMP. I'd never thought about it that way. What a shame to be hindered by such a terrible name choice (in GIMP's case).

[TOGoS]

I'm always surprised at how quickly people will dismiss something just because of a name. I once wrote a bug tracking system/wiki for my company to replace the terrible one that we had been using. It did everything we needed and was much nicer to work with, but the reason people gave me for why we never used it was that the name sounded funny. It was a made-up word, totally an afterthought. sigh

[busymom0]

What was the name?

[gnrlst]

I am a big believer in "nomen omen" -> latin for: destiny is in the name. It's obviously an exaggeration, but to your point, this is one of those cases where the name influenced how the tech was being used (and in what environments), regardless of its underlying potential.

[mananaysiempre]

“Nominative determinism”[1]

[1] https://unsongbook.com/

[resoluteteeth]

I'm not sure the problem is the name.

I think there has been a lot of recognition in the last 10 years that sqlite is actually quite robust, but it still hasn't been considered suitable for serious use is based on how software and database servers have traditionally operated.

It seems like what's changing that now is the recognition that other approaches may make more sense given modern software architecture.

[kevincox]

I don't think stability is the main concern. However for the longest time SQLite didn't have great support for multiple writers and it is still pretty basic. So it works really well for smallish numbers of clients or write-light workloads but if you want to pound it with inserts and updates it still isn't as good of a choice as other RDBMSes.

[ternaryoperator]

> For much of its lifetime, SQLite wasn't taken seriously

As far as I can recall, it was always well regarded as an embeddable database. What makes you think that it wasn't taken seriously "for much of its lifetime"?

[aaaaaaaaata]

By standard web devs.

Been in missiles for a while.

[_the_inflator]

I look at it differently. In my opinion Android started popularizing SQLite. For me kudos go out to Android, for integrating it on a large scale.

[thro388]

There were other embedded databases; Firebird, Foxpro etc..

[sethev]

It's hard to imagine a timeline where SQLite was more successful.