Hacker News new | ask | show | jobs
by jerglingu 1260 days ago
Observation/mini rant: unfortunately the data industry is both very susceptible to fads and hype, and does not have widespread standardization or a generally-accepted set of best practices. The subset of data "influencers" whose discourse occurs mostly through Twitter and Substack have especially heavy sway in what is the current Data Big Thing. These people introduce new buzzwords and ride them to the mainstream, or redefine what were previously more-or-less anchored down concepts into new interpretations. It feels tepid and arbitrary, almost postmodern. On top of this, much of the "thought leadership" is being driven by individuals who lean heavily towards the soft-skill side. So we have a heavy overindexing on strong opinions around organizational methodologies, team structure and roles/responsibilities, and other Big Ideas without much engineering representation or consideration. What spawns from this are things like the "data mesh" and the bastardization of both data concepts and clear communication/thinking. The data mesh white paper perfectly encapsulates this[1]. I challenge anybody to try to read it and understand what the hell the author is even vaguely trying to say after a few passes.

[1]: https://martinfowler.com/articles/data-mesh-principles.html

1 comments

I do see what you are saying, though I think a lot of what they say makes somewhat more sense if you are familiar with their consulting "bubbles" and the language used within them.

But the problem exists even on a concrete technical level. Look at how many different "big data" products are out there. The Apache project alone has probably 15 different tools that largely overlap.