Hacker News new | ask | show | jobs
by qwrusz 3543 days ago
I encounter this a lot at work. People needing more advanced stats skills for a new role and not having much training in it or if they did it was years ago. (I work in finance which has become increasingly quant and stats heavy - faster than training in it has).

My advice: Figure out exactly what type of stats work your teams are doing. Make a list of those topics. Random example: are those Kolmogorov–Smirnov or Mann–Whitney tests? Then hire a tutor who knows that stuff - maybe a grad student somewhere, can be remote over skype even.

If you are not 100% sure what you are looking at at work and what to put on this list of topics...hire a tutor and show them stuff from work (if the work is proprietary/confidential, recreate it with dummy data or just give rough examples) and ask what topics would be needed to nail one's understanding of this work.

Statistics is a huge subject and if you buy a textbook you may spend a ton of time on stuff that's just not relevant when you could be going a bit deeper into a sub-topic that is very relevant to your work. Also a lot of what looks like statistics is actually found under applied math books/courses not statistics.

Lastly, in case this needs be said, after you get the basics on a stats topic, the most important question to ask a stats tutor is "where do people usually fuck up when doing this?"

Stats in practice is often more about not making errors than it is about accuracy. Find out where people often fuck it up, especially as a manager and 2x as they are not statisticians either it sounds like.

2 comments

"Stats in practice is often more about not making errors than it is about accuracy."

What's the nuance? (Serious question)

TL;DR What to focus on? Some statistics advice: Don't be a liar. Don't be a biased idiot. Don't fuck up. The software should handle the rest.

Nuance is a fair serious question. And this could easily turn into a debate of semantics or philosophy (will add links at bottom tho^).

But what I meant was statistics in practice isn't about proof of some truth but about chance of disproof. An analogy in jurisprudence: there is a difference between "not guilty" and "innocent".

Someone may or may not be "innocent". There's even presumption of innocence. But then in practice, lawyers give evidence to a jury to decide beyond a reasonable doubt if someone is "guilty" or "not guilty".

What's the focus? It sure looks like the work is more focused on "not guilty" vs "innocent".

Furthermore, in statistics there are errors...eg statistical errors, random errors, systematic errors, type 1 errors, non-sampling errors...lots of errors. You can't eliminate them. But you can be aware of them and reduce them where possible.

Now, statistical software deals with errors to the extent statistics techniques exist and the technology can handle the process. Sort of like spellcheck.

But software can't fix everything. Most importantly it can't fix if the person using software is an idiot.

Too many times I have looked like an idiot for sending an email where spellcheck put the wrong word. What to do? I could write a new algo to make spellcheck better or I can just double check the email next time.

^Links to semantics and philosophy stuff: Some fields try to have precise, official definitions for words like "error" and "accuracy".

See ISO 5725 or longer list of examples on wikipedia: https://en.wikipedia.org/wiki/Accuracy_and_precision

Of course, philosophy also addresses the nuances. Way more fun to read than ISO technical documentation.

Short list of philosophy of statistics issues on wiki: https://en.wikipedia.org/wiki/Philosophy_of_statistics.

Better, longer list, which is worth reading as it includes more interesting and broader philosophy of science issues: http://plato.stanford.edu/entries/statistics/

If lists of philosophies are overwhelming and you want one random example of it...What is the probability the sun rises tomorrow?

Long post. Lastly, a joke: 'A physicist, an engineer, and a statistician go duck hunting. They spot a duck in the distance and the physicist takes the first shot, but just misses left. The engineer shoots next, but just misses right. The statistician yells, “we got it!”'.

[Edit] At this point I might as well add Buffet's 2 rules for investing: "Rule No. 1: Never lose money. Rule No. 2: Never forget rule No.1”

Thanks. Any idea how to go about finding a tutor?
Where you located? Likely a local tutoring company can find one for you at the skill level you need.

Others on here might have input on websites that do this.

We use pretty advanced stats at work so I cold emailed a professor at a local university and he was able to connect to grad students (they get these types of emails more than one thinks).

If interested, if I remember it was ~$120 an hour, the tutor had a PhD already and was doing postdoc. 3 hours a week with a tutor combined with self study at home for about 2 months before feeling "ready".

The self study never stops of course.

PS. If you think it would help and if you are able to post more info about the type of work your guys are doing, I'd be happy help answer questions you may have.