Hacker News new | ask | show | jobs
by niels_bom 868 days ago
Would you say it's slower than file IO too?
3 comments

It’s not slow by itself. It’s a single point of bottleneck that will inevitably become slow as you cram everything into it.
...but by trying to avoid the bottleneck and moving things to backend, you make things 10x worse resource wise for the DB. So it is not a easy tradeoff.

Take any computation you can do in SQL like "select sum(..) ...". Should you do that in the database, or move each item over the network and sum them in the backend?

Summing in the database uses a lot less resources FOR THE DB than the additional load the DB would get from "offloading" this to backend.

More complex operations would typically also use 10x-100x less resources if you operate on sets and amortize the B-tree lookups over 1000 items.

The answer is "it depends" and "understand what you are doing"; nothing about it is "inevitable".

Trying to avoid computing in the DB is a nice way of thinking you maxed out the DB ...on 10% of what it should be capable of.

Yes. Aggregations and search are often best done as close to the data as possible, in the DB.

Rendering html, caching, parsing api responses, sending emails, background jobs: Nope.

Basically, use the database for what it’s good at, no more.

Well, it is file IO, plus processing on top. But it's not that simple, since if your data is small enough it can all be loaded into memory, allowing you to sidestep any file IO. But you still have the processing part...
Kind of irrelevant since a DB provides some guarantees that a simple file does not by default.
GP was responding to a comment comparing it to network IO in terms of bottlenecks in your application stack ...?