| > What bothered me about your comment was that it sounded a bit like "wow, I found that RDBMSes suck at this specific type of computation that they're not built to deal with, therefore query optimizers suck in general", which seemed like an over-reaching argument. Gotcha. Yes, it was more "I have some things I need to do, and RDBMSes can't do some of those things, which rules them out as a solution". There is for sure lots of great stuff in query optimization, and it makes some queries lots better. > I guess what I took away from his claim was that the contribution of MR itself was not the problem, but the fact that while creating that model, they ignored a lot of other learnings: e.g. blocking operators can be detrimental, indexes are handy to have. Plus the fact that everyone /else/ who didn't have Google's reasons to forego all those niceties still dove head-first into "let's use MR for everything". They did not ignore them, they just weren't building a database. MR was much more a scalable HPC replacement than a data management product. The main reason that the DB community took a huge step backwards is that they (incl Stonebraker) had doubled-down on mediocre compute abstractions, and found they needed to revisit much of what they'd done, because it just didn't work. > Ah, for me the main question and discussion above was "are RDBMS techniques even relevant at this point", to which my answer is yes, absolutely. That doesn't mean you have to take every concept from it wholesale: many techniques that developed in one context are applicable in others, regardless of Stonebraker's opinions. I totally agree with you that they are relevant (and am active in the area). Stonebraker takes a much stronger position, and the appeal to his authority was what triggered me. I didn't mean to point that at you as much as it may have turned out. |