Hacker News new | ask | show | jobs
by maxerickson 4025 days ago
How do you measure if it works or not?

I mean, if you only talk to men and then measure where you derived utility, I'm not sure you've properly evaluated the model yet (if you interact with a certain percentage of women at conferences and keep track of all this in order to make sure that your model is working out properly, well then, more power to you).
1 comments
yummyfajitas 4025 days ago
I really can't write an entire textbook on Bayesian decision theory in an HN comment. Honestly, if I could, I'd write the actual textbook - we really need a good one. I'm not claiming to have a rigorously tested model of exactly who to network with at a conference. I'm claiming that ignoring base rates results in worse decisions.

And if I made this point in a non-political context - e.g., "you must account for the base rate for $disease when interpreting a $disease test" - no one would be disputing it.

If you or anyone else here can even present a (utilitarian) model where ignoring base rates leads to a better decision, by all means do it. I'd love to see this, though I suspect the actual outcome of such an effort will merely be the person attempting to do it gaining a much better understanding of Bayes rule.

link
maxerickson 4025 days ago
My issue isn't with the point that the base rate matters, it's with the mathematricality.

And I do think that is fair, if there is not a simple way of actually measuring the utility and such (re your complaint about $diseases, medicine has at least somewhat reliable tests), all the stuff about the modeling is just theatrics.

link
yummyfajitas 4025 days ago
Am I correct in interpreting this post as saying "I agree base rate matters, I just wish you didn't provide a toy example illustrating that?"

Also, I now strongly recommend you go through the exercise I suggested. Then you'll realize that accurate tests do NOT somehow eliminate the need for models or accounting for base rates.

link
maxerickson 4025 days ago
Also, I now strongly recommend you go through the exercise I suggested. Then you'll realize that accurate tests do NOT somehow eliminate the need for models or accounting for base rates.

I'm reaching for the opposite point. Some sort of meaningful evaluation of actual outcomes is necessary to make utilitarian decisions. If the evaluation of the outcome is arbitrary, then so is the decision.

link