I purchased this book on sale at the Kindle store to get a detailed sense of what his methodology was, and it seems he just uses Google Trends to compare search volumes across locations combined with Google AdWords to give the data a numerical scale and break down the data demographically. It seems statistically dubious at best, especially the part where he makes assumptions based on search autocomplete results (which vary based on location, IP address, browser, OS, and other factors).
You can read more about his methodology in his papers here [1]. He wrote his PhD thesis on "Essays Using Google Data". I can't think of a better way to do these estimates with the limited data provided, but I'm just not convinced it's a strong enough proxy for the conclusions he's making.
Bear in mind that he had a look behind the curtain - working at Google and talking to the relevant people about the results does give reason to believe that he knows more about what he's talking about than a dilettante on Google Trends. And if there's anything that economics PhD does you for, it's a large dose of statistics.
> assumptions based on search autocomplete results (which vary based on location, IP address, browser, OS, and other factors).
I imagine you could just do the same searches in an Incognito browser window and get results not tailored to you, nor your location, IP, etc. (Or, I would be rather surprised if google searches were STILL tailored to some determinable attribute of you in an incognito browser window.)
I also imagine Seth did this (but only because I would be very surprised if he did not).
Incognito mode doesn't do anything to hide your location or IP (basically the same thing on non-mobile). It's really just meant as a porn mode: nothing you do is saved to disk.
I'm pretty sure that incognito mode still gets you ip-based and location-based personalization -- try experimenting at your workplace with searches that you think your co-workers might use.
This is a well understood phenomenon in the world of Garbology [1] called the Lean-Cuisine Syndrome [2]
In one study garbologists found that people uniformly underreported the quantity of junk food they eat, and overreport the amount of fruit and diet soda they consume. Most people also underreport their consumption of alcohol by 40 to 60 percent; on the other hand, heads of households regularly exaggerate the amount of food their families consume. [3]
Or the well known phenomenon that when people are asked how many lifetime opposite-sex sex partners they've had, the average reported by men is about twice the average reported by women (though, by construction, they ought to be the same - just imagine a bipartite graph (undirected!) and count the lines going from left to right and compare with the number of lines going from right to left).
See eg. David Spiegelhalter, Sex by Numbers: What Statistics Can Tell Us About Sexual Behaviour, chapter 3.
Do you recall when you last read in the newspapers about how someone convicted of murder has their searches used as evidence against them?
What comes to my mind when I read this is what the base rate of suspicious searches is. If 10% of days include at least one suspicious search in the base population, the value of such a search as evidence of malice is greatly reduced. But most people don't consider base rates.
On the topic of the differences in parents' concern for their boys vs their girls, for genius vs overweightedness, there's a simple rationale: for future career and relationship success, a boy's intelligence and a girl's looks are relatively more important than vice versa. It's not right or fair, but it's believed to be true, and that's what makes people search for it.
"When we lecture angry people, the search data implies that their fury can grow. But subtly provoking people’s curiosity, giving new information, and offering new images of the group that is stoking their rage may turn their thoughts in different, more positive directions."
This was the most interesting takeaway for me. It's one thing to know that negative behavior is underreported on surveys, but much more valuable to draw potential conclusions about how the situation might be improved.
I'm not sure that counting gay porn searches (example they gave in the article) is a reliable indicator of whether the seeker is gay. This may be TMI for HN but once in a while I (a straight male with zero attraction to men ever) have searched for and "enjoyed" gay porn.
I was curious about this phenomenon and found many forum posts where straight guys say they've watched gay porn.
What scares me is that it would be possible for a third party to identify people who perform searches for gay porn (in this example).
This could be done by setting up an ad campaign directed at gay people, but selling something innocent. Only people who searched for gay porn will see the ad, and so the third party can determine that those buyers have looked for gay porn.
I'm going to go out on a limb and suggest that the occasional hetero guy searching for gay porn is a rain drop in the deluge. If gay men search for gay porn as often and as reliably as hetero men search for hetero porn, the occasional hetero guy conducting a bi- or gay-curious gay porn search would be insignificant.
Might that mean that you are bi? I mean, if you like watching it you might enjoy doing it.
If your first response to that suggestion was visceral, maybe you actually have some aversion to the idea that you are. It makes sense, the world is easier for straight people, but it might bear closer examination.
This does not necessarily follow. I have (ahem, my username is not anonymous but I don't think this is controversial anymore) enjoyed porn of a nature that I wouldn't necessarily participate in. You can also enjoy porn that is completely fantastical and thus impossible to participate in, such as the infamous "hentai", or (I will never understand this) that "my little pony" porn genre, or from an unrealistic (or is it... fantastically realistic?) flash game such as "Super Deepthroat" (introduced a while back to me by a female Redditor who claimed she could ONLY orgasm to it, perhaps surprisingly), and if this article is to be believed, human women are generally turned on by ANY sexual expression, even if they claim not to be, which would ostensibly include scenarios they would be unwilling to participate in: http://www.nytimes.com/2008/06/12/fashion/12bisex.html
(To any future employers, friends, clients, interlopers and all others who encounter this after getting to know me... None of this should terribly surprise you. ;) )
Obviously, it does not follow, but it seems likely that there is a correlation. I think it bears examination because knowing yourself is generally a good thing.
I say this because I too enjoy some porn that I thing I wouldn't want to participate in. I found it rather productive to consider what this means about me.
Could be, but that's just one option. Another one being: purely curiousity. Engineering (and other) minds often think in ways like "What does it look like? How does it work practically?" etc, and that goes for any subject, so if that happens to be gay sex then gay porn would surely be a good starting point (while keeping in mind that the depicted stuff might not exactly represent reality, like a ton of porn out there).
Indeed, it is possible to enjoy gay porn without wanting to have gay sex. There is weird stigma around this though, which is why I suggested introspection. The stigma might distort some self-perception.
It is hard to talk about this because sadly 'you might be gay' is often read as an accusation.
I occasionally enjoy gay porn as well and even had a "gay experience" when I was a teenager (I wouldn't say it wasn't pleasurable, but it wasn't amazing either).
That being said, I am definitely not gay and don't have any further desire to have sex with other men; what can I say other than human sexuality is complicated?
I agree, I don't believe that someones search habits are indicative of their sexuality. Just because a woman watches lesbian videos or a man views gay porn does not mean that they are gay. Maybe they could be slightly bi-curious but still have no interest in a relationship with the same sex. And how do they even know that the searcher is a man? Lots of straight women watch gay porn and that obviously does not make them gay.
How about Google revealing some of their own darkest secrets instead, that would be interesting. There are clear incentives for the ruling powers to make everyone believe we're all as messed up in our heads as they are, makes us more forgiving and accepting of whatever leaks out from their filthy games.
I use google search all day every day. On those occasions when I want to do a search and not have it end up in advertiser databases, I switch to duckduckgo.com.
Do you HN'ers generally consider DuckDuckGo to be trustworthy for search queries of a private nature?
I personally take them at their word about privacy -- there doesn't seem to be much choice to do otherwise. And it's easy to add !g to a search to send it over to google when better search results are required. Actually it's really convenient to be able to do !gm for google maps, !wa for Wolfram Alpha or use any number of other bang commands.
There is a PR fluff piece about this book on HN every week. -1 spam.
In the US version of this article had a big focus on apparently secret attraction for overweight women. I wonder what the "darkest secret" is that pushed the author to focus on whether people are gay or not in the U.K.?
The article suggests that racist jokes are a problem to be solved, but racist jokes serve a noble purpose according to Slavoj Zizek: https://www.youtube.com/watch?v=IISMr5OMceg
What a crock of utter and complete nonsense. If you isolated the simple statements of fact then this article would still be a long and enjoyable read, but instead i am assaulted with corresponding insane ranting; accusations against huge swaths of people that are totally unsupported. So nobody googles if their daughter is smart? Maybe its because parents are worried that looks and not intelligence are important for cohesing in school and not getting bullied, for girls. Maybe the same for boys, but with social quickness and intelligence. Maybe they are responding to the behaviour of the kids themselves, who might openly worry about being smart or ugly; perhaps girls and boys have their own tendancies about which one worries them because of the small fact that they are differnt biologically, and that visual attractiveness is much more important for men? Not a single alternative is explored by this author and it should be no surprise because when you zoom out and look at the whole picture of this article its obvious that this guy is simply using these issues as a tool for popularity and self advancement. Im not saying it shouldnt appear on hn because the phenom of using these issues for personal gain and the death of journalism is in itself an intellectual curiosity. But i will die before letting this shit go uncommented upon.
> accusations against huge swaths of people that are totally unsupported.
The search data is meant to support the accusations. A main point is that other methods of supporting such statements (like surveys) are unreliable.
> So nobody googles if their daughter is smart?
The article doesn't say that, it says parents are about twice as likely to google if their son is smart vs daughter.
> Maybe its because parents are worried that looks and not intelligence are important for cohesing in school and not getting bullied, for girls.
This seems like exactly the problem the author wants to bring up and address. Society values appearance above intelligence in women. This is apparently also true in how parents see their children.
The author doesn't draw conclusions about this prejudice, he is merely remarking that it exists.
While we can blame society for it, we can also see the same pattern in dating strategies of men and women. Men selects mates on appearance, causing social status for women to be primarily based on appearance. Women selects mates on wealth, causing social status for men to be primarily based on having or the ability to raise money. A theory would thus be that parents that want their child to have the highest social status will among other things try to maximize those attributes.
The author conclude that this mean there is a bias against girls.
> Society values appearance above intelligence in women. This is apparently also true in how parents see their children.
Sure, parents who are interested in their children's well-being will also be interested in what society thinks of their children.
This data seems like a weird source for the claim, though; I'll bet the people searching the internet for "is my daughter smart" significantly outnumber the people searching for "is my daughter pretty".
"is my daughter smart" and "is my daughter pretty" both show up as 0 in google trends (as I search today). What does "daughter pretty" include? I would predict massive public interest in questions of the form "is Reese Witherspoon's daughter pretty?"
Yeah, they don't show up with a zero, it just says not enough data to show so hard to draw any conclusions. I agree this is an imperfect proxy and it's not definitive but I'd still absolutely take you up on that bet (conditioned on there being some way of actually resolving it).
But maybe the real question is, why aren't people asking "Is Reese Witherspoon's daughter smart?"
I can honestly say it's never even crossed my mind to do a web search on a phrase like "Is my wife cheating" or "Is my son gifted." I really can't see finding the answers to questions like that on the internet.
>What a crock of utter and complete nonsense. If you isolated the simple statements of fact then this article would still be a long and enjoyable read, but instead i am assaulted with corresponding insane ranting; accusations against huge swaths of people that are totally unsupported. So nobody googles if their daughter is smart?
Millions do. Just 2 times less millions that people who wonders if their sons are smart. Which was explained in depth in the article.
>Maybe its because parents are worried that looks and not intelligence are important for cohesing in school and not getting bullied, for girls. (...)
This sounds more like a "totally unsupported" claim that what the article suggests. And even if that was the case, it would show the same stereotyping done by the parents as the article already suggests.
Not a single alternative is explored by this author and it should be no surprise because when you zoom out and look at the whole picture of this article its obvious that this guy is simply using these issues as a tool for popularity and self advancement. For example on those occasions when I want to do a search and not have it end up in advertiser databases, I switch to duckduckgo.
You can read more about his methodology in his papers here [1]. He wrote his PhD thesis on "Essays Using Google Data". I can't think of a better way to do these estimates with the limited data provided, but I'm just not convinced it's a strong enough proxy for the conclusions he's making.
[1] http://sethsd.com/research/