"Big Data" and "Data Scientist" are the latest buzzwords like Web 2.0 and Java were during their ORA hype-r eras.
There is a lot more data. There are a lot more tools and technology to deal with data. There is a need for high quality people that understand how to handle data. But there are no rockstars, there are people that have passion and spent years learning and teaching their craft. But they aren't rock stars. The best may get paid as much as a very well off doctor or business owner, but they aren't going to fill stadiums around the world, sell millions of t-shirts or be targets of media gossip columns.
My main objection to it is that this rebranding of statistics (and machine learning I suppose) obscures the deep political connection statistics has[1] and in so doing obscures the fact it's going to have major political ramifications across this century. Happy clappy summer of code donations from Google to EFF or the Sunlight Foundation are entirely not proportional to the extremely high level of abuse that could potentially stem from these technologies (see the use of IBM sorting machines in the Holocaust).
Technology at a basic level amplifies agency, certain techniques and their resultant technologies have benefited individuals and citizens more than State agencies and with others it's been the inverse. I think we are moving into an stage where there is potential for massive recentralisation of control in various domains.
But this is HN, not a political discussion forum, most of the 'frighteningly ambitious ideas' to grace these pages are to do with pushing out more ads or other such pablum. PG isn't a philanthropist and YC isn't a charitable foundation; HN is naturally aligned with those objectives.
Thanks for this. It's a humbling comment that puts reality into perspective: there are several ethical and moral challenges we coders are gonna be put through in this century, far, far removed from the Silicon Valley bubble and it's love of ads. And it's this love of ads (read, millions and billions of dollars) at the expense of enriching corporations and tearing down privacy that worries me a lot about our future, if it's going to depend on programmers' current moral standards--though really, it's all human nature, it's not like we're a different breed. But we definitely are more informed than the average citizen, and more of us should take responsibility for increasing freedom and basic human conditions, as opposed to selling social local deals.
As long as we're throwing buzzwords around, why not [self post plug alert!] switch over to Analytics Syndication Services? I've envisioned it as a paradigm shift to align your core competencies. http://tech.theswamp.in/post/2013/04/01/analytics-syndicatio...
that sounds like an awesome slope to slip down on. let's get rid of all of those ridiculous terms.
how about we just use the oh so boring "professional" instead? i want respect from society for my work, not a cute pet title co-opted from other industries, where they can use these terms with a straight face and without rolled eyes from the audience.
What troubles me most about the title "Data Scientist" is that it really means "Statistician" or "Statistical Analyst". There are so many interesting things you can do with lots of data, stats is but one of them. What do we call people who are good at some of the other disciplines?
I've spent bits of my career working with fairly large data sets at one time or another, and providing discovery, insight and analytic tools into that data, but very little of it seems to have anything remotely to do with what the job descriptions for "Data Scientist" are asking for.
Consider this, building a very large graph of the internet, then using various models on that graph to find unique and actionable insights: such as finding routing bottlenecks for a video delivery service, involves lots of data, lots of scientific like exploration, yet isn't a "data scientist" job by the job reqs.
How about this, building a text parser that can finely categorize and make recommendations for a research organization based on millions of grant proposals, all categorized into various "mission silos" that research organization is built around. Not a "data scientist" job.
Analyze multi-lingual news stories to build a real-time alert system for conflict analysts. Not a "data scientist" job.
Building a tool that can scan multi-spectral aerial imagery and automatically extrapolate man-made structures from natural, catalog all of the different vehicle makes and models, and generate a predictive model of commuting patterns, or make recommendations for housing development based on perceived socio-economic conditions? Not a data-scientist job.
Collecting information on who propositions who from a dating web site, normalizing the data for population and writing a report on the findings? That's a "data scientist's" job.
It's not that that kind of work isn't valuable, only that there are so many other kinds of things that involve what might intuitively be called "data science" that calling just the one discipline "data science" is doing a disservice to what should be an amazing discipline -- part Computer Scientist, Part Analyst.
This is a pretty fluffed up article that missed the wave of "data scientists are awesome" articles last year.
But I think it's important to not underestimate the shift that has been happening. The race to automate tasks has always been accelerating, but it hit an inflection point a few years ago when mainstream business people realized what current technology can achieve. Pretty much every industry I've seen has been or can be drastically transformed by better data management and predictive analytics.
It's going to affect all of us, so I think it's worth following closely.
I agree. The article was meant to be a high-level introduction to the phenomenon for the Dutch audience. They missed that wave. I had know idea it was going to end up here:) Apparently a controversial title can still trigger lingering emotions:) Most people still don't seem to have a clue what is possible, though.
IMO something is accelerating/changing and we hit an inflection point. It does help to give it a denominator. It's human, not just marketing. I'm sure one time people thought compsci and software engineering were redundant or pretentious. I'm still ambivalent about the name data scientist.
Data Science and "Big Data" have done a really good job of marketing themselves. As a result, people recognize that these roles are valuable, Data Scientists get paid & treated well, and it tends to work out well for everyone.
What can we learn that we can apply to technology in general? I feel like "programmers" and "IT" tend to be undervalued while Data Science tends to be accurately or even overvalued.
> What can we learn that we can apply to technology in general? I feel like "programmers" and "IT" tend to be undervalued while Data Science tends to be accurately or even overvalued.
Given that Data Scientist is just a sexier rephrasing of Analyst, my suggestion would be to retitle Programmers as Computational Engineers. Or use Lisp.
It is confused to give data science much credit for the success of Netflix. Netflix succeeded because of content, wide platform availability, and solid marketing. They ran a million dollar prize contest that ultimately did nothing for their code base, but did generate a lot of positive press.
"Big Data" and "Data Scientist" are the latest buzzwords like Web 2.0 and Java were during their ORA hype-r eras.
There is a lot more data. There are a lot more tools and technology to deal with data. There is a need for high quality people that understand how to handle data. But there are no rockstars, there are people that have passion and spent years learning and teaching their craft. But they aren't rock stars. The best may get paid as much as a very well off doctor or business owner, but they aren't going to fill stadiums around the world, sell millions of t-shirts or be targets of media gossip columns.