[ecaron]

Can someone explain why this is a topic worthy of discussion? I get that they had a SEO tactic, Google penalized them, they offered an empty apology for it, and now we're up to the present.

Stating the obvious of "traffic drops when Google penalizes you" seems like a "sun rose again today"-type story.

[ruswick]

Why? Because the sheer magnitude is crazy. Rap Genius's traffic fell by nearly two thirds! This is a very prominent, YC-affiliated startup that is being nuked by Google. It should come as no surprise that the news is seen as significant by the HN crowd.

Moreover, I think many people are upset with the scope of Google's punishment. De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted. Even searching directly for the name of the site does not yield any Rap Genius results.

If I type "Rap Genius" into Google, I am justified in expecting Rap Genius to be the first result. Period. When Google goes out of its way to effectively expunge a site from existence (to the extent that Google can expunge a site), they are doing a massive disservice to their users and are degrading the quality of their search results.

There is no reason that a response of this magnitude by Google is justified, and by issuing such a draconian response, Google has lessened the usability of their product and done a disservice to their users.

[zach]

Penalties are blunt instruments.

The whole point of this penalty is that Google cannot trust the signals that are in its index when it comes to Rap Genius's site. And it sure isn't Google's fault that they can't.

Can Google subtract out all the "naughty stuff" and decide if Rap Genius deserves to even be on the front page for "rap genius" anymore? I doubt it.

If Google's search had a category for "obvious searches" then yes, maybe they could apply a penalty as delicately as possible. But delicateness is not really the point. Dropping them down 50 spots is blunt and simple. I think everything about this approach makes sense.

[loceng]

From I what I heard the penalty is only for a month long? Still, what other practices has Rap Genius been using all this time, without being caught, in order to gain so much traffic?

[GuiA]

>De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted. If I type "Rap Genius" into Google, I am justified in expecting Rap Genius to be the first result. Period. When Google goes out of its way to effectively expunge a site from existence (to the extent that Google can expunge a site), they are doing a massive disservice to their users and are degrading the quality of their search results.

Necessary reminder that Google does whatever the fuck they want with their service, and that no one is in a position to expect anything. Does it suck? Yes. Is it a predictable outcome of the free market on which Silicon Valley and the tech industry in General is built? You bet.

[dragonwriter]

> Moreover, I think many people are upset with the scope of Google's punishment. De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted.

If the impact wasn't significant, it wouldn't be a disincentive to engage in black hat SEO.

> There is no reason that a response of this magnitude by Google is justified

Yes, there are reasons. The reasons are "specific deterrence" and "general deterrence".

[joelrunyon]

Cutts has said before their goal is to "break the black hatters spirit" (http://searchengineland.com/googles-matt-cutts-on-search-spa...). I assume that's what they're trying to do here.

[chris123]

The response was designed to send a message. And it seems to have accomplished that.

[belorn]

In one of the other threads, some stated that Google do not have monopoly status. The claim they used was that google only has 25% of the search market.

This topic is interesting because it shows how much in a monopoly status google really is in in controlling where traffic goes on the net.

[gojomo]

Interesting point!

Google's search team has been famously resistant to any influence by Google's sales/marketing teams on ranking decisions. Search quality gave rise to the 'golden goose', after all, and the self-conception of the team, and its individual members, is based on independence from purely money-grubbing considerations.

But I wonder, what happens when Google's antitrust lawyers came to the ranking team, and say: "soften this penalty [or otherwise tweak these quality-rankings], because otherwise we'll be in more trouble with regulators or leave too much evidence of market-power."

Could the search team still say, "buzz off, we're sticking to what we know from the numbers is what's best for our users"? Or does keeping regulators happy, and avoiding smoking-gun fact-patterns, trump everything? I suspect it often will, because of the unique risks of state enforcement and appearing unlawful. It's also easier to rationalize "we're sacrificing our own idea of what's best because the law seems to require it" (even if the actual risk is fuzzy before losing in a formal legal process) than "we're doing this to make the guys over in ads/mobile/etc a bit more money this quarter".

[angryasian]

you are not using that word correctly. If Google was the only company doing search and prevented others from entering the market you would be correct. Being the best through a superior product and innovating is not a monopoly.

[gojomo]

Doesn't matter how you achieve a monopoly: it's defined by dominance, not bad-behavior. And once you have monopoly power, things that used to be legal for you to do – for example certain pricing or bundling strategies – can become illegal.

[crazy1van]

> Doesn't matter how you achieve a monopoly: it's defined by dominance, not bad-behavior.

Well when you describe it like that, makes me think antitrust laws are just silly. Isn't the job of a company to be the best and beat their competition?

[angryasian]

the entire argument is questionable as to what Google's strategy is but they are only in that position because of a superior product in a market with a relatively low barrier.

[gojomo]

Low barrier?! Giant companies like Yahoo, staffed with large teams of search technology pioneers, have been scared out of search competition. Google touts its massive custom infrastructure. The arms race between Google and SEO creates a massive pollution backwash of false quality signals on the web, that make things way harder for any upstart which lacks Google's decade-plus of proprietary know-how and billions in hardware.

And that's before considering the contractual defaults across software (Firefox/toolbars/bundleware) and subsidized hardware (Android), that few people change. (Many don't even know it can be changed, or don't even know the difference between "Google" and "the Internet"!)

Those are all big barriers to entry, even if they've been earned by excellent technology and business strategy.

[angryasian]

while DDG started by 1 guy and entrants like blekko or qwiki can differentiate in other ways and enter and compete easily.

Again everything you are saying is because of superior product/employees and innovation. Your entire argument is based off no one else can come up with a better product so lets punish them

[bluekitten]

From the perspective of web users, there is choice(even if heavily skewed by default search in browsers). But from the perspective of web sites/web companies/web devs/web advertisers, there is very less choice and bordering a monopoly, as this example illustrates. Even worse if you're someone like MapQuest, a shopping site, or Yahoo Finance, since most people tend to search locations on search engines/browser and are quickly steered to their equivalent competitors from the search engine company. Anyway, usually antitrust regulations are about market power and influence, for example see DoJ vs. Apple over eBooks, even though iOS never even had majority share, forget about a monopoly.

[angryasian]

The U.S. accused Apple Inc. and five of the nation's largest publishers Wednesday of conspiring to raise e-book prices. I don't even see how thats relevant to how Google does business within their own ad products. Who exactly is Google colluding with to suppress or hurt competition ?

Companies will do business where the users are. The minute there is a better product or users abandon google, those same people you listed will leave as well. So maybe we should go after Facebook for destroying MySpace as well ? The entire argument is based on the fact that Google offers a superior product and should punish them for this.

[dragonwriter]

> This topic is interesting because it shows how much in a monopoly status google really is in in controlling where traffic goes on the net.

Does it? I mean, is there any reason to believe that Rap Genius is typical in their degree of dependence on Google here?

[res0nat0r]

I'd replace the word monopoly with popular.

[belorn]

"A monopoly (from Greek monos μόνος (alone or single) + polein πωλεῖν (to sell)) exists when a specific person or enterprise is the only supplier of a particular commodity. Monopolies are thus characterized by a lack of economic competition to produce the good or service and a lack of viable substitute goods. The verb "monopolize" refers to the process by which a company gains the ability to raise prices or exclude competitors." - Wikipedia

You can monopolize an market by popularity, and thus create a monopoly. However, monopoly and popularity is not the same word.

[res0nat0r]

Google is not the only supplier to internet searches and is not excluding competitors. They are merely very popular and are thus not a monopoly.

[throwawaykf03]

You are interpreting "monopoly" literally. In the context of antitrust law, you only need to have a big enough market share to appear on regulators' radars. The actual market share threshold varies from country to country, but typically it's around 60%. Google certainly meets that criteria in many countries, not only for search but for online video as well.

As for "excluding competitors", Google is having a tough time with antitrust regulators in the EU for exactly that. For a more obvious example, see their shenanigans with YouTube and Windows Phone.

[crazy1van]

I totally don't buy that Google has a monopoly on search. Anyone here could start a search engine tomorrow. Sure, it might suck, but there is no barrier to entry. It isn't like Ma Bell telephone companies who had blocked other competitors from running copper wires to homes via tons of local ordinances... which reminds me, its pretty hard to have a true monopoly without some help from the government.

[snowwrestler]

I think folks are interested to see how much the penalty hurts.

[wozniacki]

I think people - foreigners and valley outsiders for the most part - are often genuinely clueless about the workings of the valley.

They are quite apprehensive of the degree of even-handedness that is at play in the valley. They suspect an air of nuanced in-group favoritism, within startups here, that help propel some and sink others.[1]

Hopefully these kind of actions (on behalf of Google) help dispel such notions.

[1] http://www.slate.com/blogs/future_tense/2013/09/13/silicon_v...

[bushido]

I personally know of a number of legitimate sites that were invariably affected due to content and blog farms harvesting their content when Google's panda algorithm rolled out.

Those penalties hurt a lot of good people!

Anyways, this one is a bit more complex than how much the penalty hurts, rather it's because, a well known and well funded start-up adopted shoddy traffic building practices that were unacceptable years ago, got caught and then had swift and targeted judgement bestowed upon them.

However much people understand that sometimes business needs to be conducted in a grey area, there is a certain amount of satisfaction when the system catches up and behaves fairly.

[michh]

What's a legitimate site? Plenty of sites most people would call "legitimate" like high profile blogs or ecommerce sites engage in activity that's frowned upon by Google. The fact that some unwanted behaviour was no longer rewarded isn't exactly a penalty.

What RapGenius (also a legitimate site in the sense that they're not scammers and providing an useful service) did wasn't just unwanted behaviour but specifically forbidden. And yes, they're far from alone in what they did. Which is exactly why Google is making an example out of them.

I know, it sucks, all your competitors doing similar things forces you to choose between taking this risk and never making it in the first place. But that's exactly why it's a good thing Google is making a stand and setting an example.

It's a move towards a level playing field where nobody is making the internet a worse place by engaging in tricks like link farming rather than a level playing field where everyone is doing it and the end user suffers the consequences.

[johnpowell]

Same here. I am in no way affiliated with Fluther other than being a user and Panda completely screwed them. I think they got screwed because Mahalo scraped their RSS feed to steal their content while providing links to the original content. So there was a thousand links going to Fluther from a shithole of a site.

[adamnemecek]

Part of it might be Schadenfreude since HN does not seem to think much of RG.

[AndrewKemendo]

I think this is exactly it.

Their affect is another example of cultural appropriation of "urban" culture by three very affluent guys (Stanford & Yale grads), which I think turns a lot of people off, including myself. As someone who genuinely grew up in that culture, it is painfully cringe worthy and just sad to see people who have no background in it adopt it. Their Disrupt interview was a genuine display of the generally awkward douchebaggery that comes from that appropriation.

So when I see folks like this do poorly because of arrogance, it is indeed schadenfreude.

[adamnemecek]

Yeah, that interview is like the definition of the word 'tool'.

[nikapius]

It's incredibly racist and classist for these privileged plutocrats to mimic the dialects of the underprivileged. The ironic and overplayed way in which they take on the pose also makes it clear that they intend it as a publicity stunt--not as genuine respect for the culture or because they fit into the culture or want to fit in. They just want money and they think this is a money making stunt.

It's one of the clearest examples of cultural appropriation and exploitation by a corporation. Really fucking disgusting.

Imagine if an American corporation wants to sell vacations to Mexico, and as a publicity stunt the CEO dresses up like a latino stereotype and adopts a Spanish accent.

This is how the Rap Genius douchebags-in-chief come off. Racist fuckheads.

[vdaniuk]

Is it genuine rage or sarcasm? It seems to be genuine but I just can't believe that.

[d23]

I don't find any of his points to be too far off. If they were a bit more modest and down-dressed I might think they had a genuine respect for hip-hop.

[gojomo]

Beware trying to assess "what HN thinks" about a topic from informal, personal scans of items. Neither the top stories, nor the longest/loudest threads, are necessarily representative of 'most readers' or even 'most commenters' – just those that show up at those times/places.

Where the especially peeved choose to spend their attention (and comments and upvotes) can make lots of people or companies seem like community bete noires – even if most of the HN audience thinks the targets are fine or even admirable, but just has less energy for anti-carping.

[fat0wl]

why is that? i assumed HN would like then since i remember a bunch of articles about RG being big RoR proponents

[adamnemecek]

I think that it's because they are not fans of their brogrammer antics. Read this thread for example (or any other Rap Genius thread on HN)

https://news.ycombinator.com/item?id=6904224

The original link that was deleted was this

http://aphyr.com/media/recruiters/rapgenius.txt

[possibilistic]

What did I just read? I don't understand that exchange at all. So Rap Genius sent a corny email to this guy trying to hire him. And...?

[dionidium]

...and he said "no thanks" because he does not approve of RG's past behavior. They persisted. And then he said no again in verse form. That's pretty much the whole story.

[adamnemecek]

I posted it more for the response on HN, not so much the email itself.

[vdaniuk]

What you've just read was manufactured outrage for fun and profit.

[logicallee]

For me, it's incredibly interesting because I did not find anything wrong with the tactic they used: in fact, it is far, far better than actual SPAM techniques they could have used instead. Or the bullshit tweaks to content that other sites use to improve SEO score, when those tweaks do not benefit the consumer in any way.

Think about the fact that there's a human in the loop, and that it's a plug or tit-for-tat. The ways that companies actually do SEO 'within the rules' is way, way worse than that.

Also as a consumer I have only been on their site < 10 times but I loved it each time. It is an actual benefit for them to gain ranking. They have a real service, and a good one.

It also seems quite genuine, from the tone of the email. Totally normal.

You might think I have some association with rapgenius, but I don't: my only association is that the handful of times I got to their site, I was absolutely thankful and totally surprised at the quality of the result.

By contrast, there is a good chance there are textual quotations I've searched for where I missed good commentary there that I would have appreciatied.

All of these aspects, especially above-all the fact that consumers are happy when they do get to the rapgenius site, that combine to make this story very interesting to me.

I see rapgenius as the good guys, and very honest, in their attempt to gain additional exposure on real content.

[mikeevans]

Not sure, but I'm sure it doesn't hurt that RG is a YC company.

[orenbarzilai]

Google just proved all that when someone breaks their laws they punish him, even if it's a YC company.

Who knows maybe because they are a well funded YC company Google decided to make an example out of them and won't forgive them so quickly.

[xuki]

Google even penalized its own product http://searchengineland.com/google-chrome-page-will-have-pag...

[ChuckMcM]

Fundamentally it is the magnitude. Web sites often manage themselves to a metric called "RPM" or revenue per thousand. It can be thousand 'views' or for people who are interested in accuracy per thousand 'visits'. Unique IPs often translate as a good metric for 'visits'.

So if your traffic is 1.5M uniques and drops to .5 uniques, then your revenue may also have just dropped 66%. If you were staffed to be cash flow positive at your previous traffic rate, you now find yourself burning cash at an alarming rate. Your response, dictated by your reserves, is either lay off a bunch of people to curb your burn rate, or delay your accounts payable, or both. If this happens at an 'inconvienient' time, it can sour acquisitions in process, or deals with initial outlays, to be blunt it can kill you.

So then here is the thing, Google has no good way of knowing how you got to your organic ranking. There is no 'what is the rank of this place without these links' sorts of computations. So there isn't a good way to judge. But judge they must, lest others abuse their algorithm, and so they do so.

Now lets speculate that they kill off Rap Genius. Overall what does that do? Who does it help? Is it justice?

Lots of thorny ethical questions, not many good answers. And no way to make restitution. A sad place to be.

[fat0wl]

eh i think this falls into the category of "if you don't care don't click the link". some people are interested in stats...

[joelrunyon]

The funny thing about this whole controversy is that they're getting more organic links to rapgenius.

When the penalty is taken care of - they may be sitting in a better position than they were previously.

[d0ugie]

If you're right, and were this actually intentional too, well, that would be genius.

But given that their traffic is maybe off two-thirds now and that it is unknown how long google will leave the lights on (weeks, maybe months), during which time this will be long-flushed out of the news cycle and it becomes harder and harder to recover, and harder to secure things like more capital when you look this stupid, that's a high-risk bet. I'd say sabotage is more likely.

[d23]

I don't think organic links matter much at this point when you can't even Google their name and find them on the front page.

[joelrunyon]

That's why I included "When the penalty is taken care of" in my statement.

[NamTaf]

You call it a "SEO tactic" here but I'm sure you'd not bat an eyelid if it were referred to as "growth hacking" elsewhere.

The problem is how much power Google holds over most of the web. You game the results and they turn their attention on you? Congrats, you're wiped out.

Essentially, there's a very blurry line between 'growth hacking' and 'being annihilated by Google' and no-one's really sure where it is. That's what's interesting here.

[obstacle1]

>there's a very blurry line between 'growth hacking' and 'being annihilated by Google' and no-one's really sure where it is. That's what's interesting here.

Except in the cases where the line isn't blurry, i.e. the well-defined set of rules about which G has repeatedly said "we'll penalize you for ignoring these!" Unfortunately the tactic RG used wasn't a blurry-line grey-area tactic, it was one G clearly said don't use.

[skizm]

It is interesting to see just how much it hurts a site like RG.

[benihana]

>Can someone explain why this is a topic worthy of discussion?

I find it incredibly interesting to know just how hard they got hit. I find it interesting that maybe someone from rap genius will be in these comments providing some extra context and insight.

Can you explain why you care if other people discuss a topic you don't think is "worthy" of discussion? You know you can ignore the story and look at the entire rest of the front page, right?

[ecaron]

>Can you explain why you care if other people discuss a topic you don't think is "worthy" of discussion?

I thought there might be a piece to this story that I was missing. I gained a lot of value from the replies to my question (esp. the conversation spawned from @snowwrestler's comment.)

>You know you can ignore the story and look at the entire rest of the front page, right?

You didn't really need this in your comment. It was a bit of snark that devalued your otherwise justified response.

[d23]

> It was a bit of snark that devalued your otherwise justified response.

No it wasn't. Comments like yours are peppered throughout HN and Reddit, and the answer is always the same: the users found it interesting. It's a democratic voting system -- get over it.

[yapcguy]

How do you know it's a democratic voting system? Just because you assume it is, doesn't mean it is. Just ask around, plenty of stories about being mysteriously shunted off the front page.

RapGenius is a YC company and always seems to get attention on HN whenever there is some kind of perceived grievance against the company. Whether it's Google or Heroku, there's always a lot of screaming and shouting.