Hacker News new | ask | show | jobs
by hodgesrm 2743 days ago
The quoted statement is not correct at least as far as analytics are concerned. Two examples:

1.) Analytics in many enterprises increasingly feed off data lakes consisting of enormous quantities of data in object storage. SQL has a part to play but it's effectively computing aggregates and creating data marts off this deeper pool of data. Data lake architecture is likely to be increasingly dominant given the enormous growth in data volumes.

2.) Machine learning is transforming analytics. This looks like the next feature likely to be absorbed into DBMS systems. SQL integration with ML is likely to be a hot topic in future systems but a substantial fraction of ML processing will remain outside the DBMS.

So SQL is going to be present widely in most future solutions but that's not the same as saying that a single relational DBMS architecture will solve all problems. It's been clear for years that ACID-compliant RDBMS have a part in this picture but it's just part.

Overall the article still seems to be fighting the SQL/NoSQL wars of the last decade. A large part of the market is moving on to other use cases.

2 comments

The most infuriating part of "NoSQL" for me was always the conflation of SQL and RDBMS and ACID. Most NoSQL is simply non-relational or non-ACID. You still need an access layer.

After all these years people seem to have finally realized that the challenge was never SQL, it was data and you still have to think about that, even if you don't use SQL.

howfuckedismydatabase.com[0] is still as accurate as it ever was.

[0]: http://howfuckedismydatabase.com/nosql/

Oh, yes, definitely.

Data warehousing and ML have different requirements and needs than your typical N-tier web app. Even streaming event data warrants a different solution. It comes back to knowing how to choose the right database for the job.

The issue is around how these technologies are marketed – grandiose claims and few practical use cases. Once the marketing material permeates the industry and some part latches on, it becomes a self-reinforcing cycle. Blog posts, books, and courses bring the information to the masses. Then companies start to adopt the tools. Then they need to hire engineers who know those tools. So more information gets published about them because that's what people want to learn to get hired.

Many engineers today will turn to NoSQL for everything because of the past few years of marketing hype (and acronym-driven-development), and that's quite a shame.

For those looking to understand how to choose the right database for the job, I'd recommend first reading "Designing Data-Intensive Applications" (https://dataintensive.net)