[coldcode]

Sometimes looking at people's stacks I wonder if we've made computing so complicated most of the time is spent dealing with stuff that is broken, and little time is left to do anything useful. Data science seems even more into this that programming in general; and sometimes you wonder if the result is actually worth all the pain.

[joaodlf]

I feel like this happens because Data Science can only work when two professional areas clash and mix: Programming and Maths. The two are very well connected, of course, but the concepts behind the maths of Data Science are much deeper than what the typical programmer is used to. Programmers need Mathematicians as much as Mathematicians need Programmers. This is where it gets hard: Programmers find it hard to implement these concepts. On the other hand, Mathematicians don't understand what good software is.

Good data analytics software can only come when these two areas learn to teach each other. Programmers need to learn maths to the point where they are comfortable enough to implement a valid solution, Mathematicians need to learn about building software that others can use.

[user5994461]

It is not my impression that Data Science mixes programming and maths. Unless in a limited field of finance where all data and analysis are maths heavy.

[joaodlf]

I felt the same when our stats were based on simple arithmetics, "sum those revenue figures", "divide that by the total amount of users", "percentage of returning members"...

It can easily spiral into, "Pearson's Correlation" or "Give me the Linear regression of the bastard".

[user5994461]

Still not hard maths. If all you have to do is apply a simple standard well documented algorithm, there is really no obstacle to your success =)

That being said. I guess that having had maths classes in my engineering degree skews my point of view, combined with working with Quants at times, who do analysis way more advanced than that.

[joaodlf]

If you are familiar with those concepts, I would count that as a big step over what I typically see in "data science". Surely a big step over what a lot of people think data analytics to be.

Like yourself, I had quite a bit of contact with maths during my engineering degree - whether I took most of it in is a different question :) (Financial Calculus nearly destroyed me).

Developers aren't typically aware of concepts outside basic statistics, and even though a lot of algorithms are readily available for everyone to implement and benefit from, how can you use what you don't know conceptually?

I guess everyone has a different experience, depends where you're working, really. I do know of quite a few shops where the push for analytics came from the tech people, mostly because companies don't employ people with the math knowledge to identify these business gains.

[user5994461]

They are really just maths algorithm, seen in maths courses or found with a quick google search.

The typical reddit developer who got a job without a degree is unaware of many many things.

The typical developer who got a job with a hardcore interview at random financial company and is surrounded by other master's and PhD. Not so much.

The typical tech company doesn't need much advanced analysis. If they could figure out how many recurring users and revenues they have, that would be a good start :D