[Matt_Cutts]

I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links.

I mean, if you're auto-generating a page that has this text: "Elcorillord.com

Common misspellings and typos: Elcroillord.com, Elcorilolrd.com, Elcori.lord.com, Elcofillord.com, www.elcorillord.com, lEcorillord.com, Elcotillord.com, Elcorillor.com, Elcoeillord.com, Elcori,lord.com, Wlcorillord.com, Elcorjllord.com, Elcorillord.coom, Elocrillord.com, Elcor8llord.com, Elvorillord.com, Elcorillprd.com, Elcorillord.cim, Elcorillorf.com, Elcorilloed.com, Elorillord.com, Elckrillord.com, Elcoriplord.com, Elcorillord.ckm, Elcorillord.cm, Elcorillord.ccom, Epcorillord.com, Elcoril;ord.com, Elcoirllord.com, Elcoriillord.com, Elforillord.com, 3lcorillord.com, Elcorollord.com, Elcorillordd.com, Elcorill0rd.com, Elcorillord/com, Elcoriolord.com, Ekcorillord.com, Elcorillord.xom, Elcorillord.co, Elcorilord.com, Elcoillord.com, 4lcorillord.com, Elcoriloord.com, Elcorillorr.com, Eldorillord.com, Elcorillord..com, Elcorrillord.com, http://www.elcorillord.com, El orillord.com, E.corillord.com, Elcorillord. om, Elcorilllrd.com, Elcorillrod.com, Elcoriklord.com, Elcorillorrd.com, Elcorillordcom, Elcorillkrd.com, Elcorillord.om, Elcorlilord.com, Elco4illord.com, Elcorillrd.com, Elcprillord.com, Elcodillord.com, Elcorillordc.om, Ecorillord.com, Elcoorillord.com, Slcorillord.com, Elcorillorx.com, Elcorill9rd.com, Elcorilpord.com, Elcorillord.cpm, Elcorillord.fom, Elco5illord.com, Elc9rillord.com, Elcorillird.com, Elcirillord.com, Elcorillord.clm, Elcorillors.com, Elcorillord.vom, Elcorullord.com, Elcorillord.comm, Elcorillord.c9m, Eocorillord.com, Elcorilloord.com, Elcourillourd.com, E,corillord.com, Elcorkllord.com, Elcorillodr.com, Elcorillodd.com, Elcorillord,com, Elcorillotd.com, Elcorillod.com, Elcorillor.dcom, Elcor9llord.com, Elc0rillord.com, Elcoril,ord.com, Elcorilllord.com, Elcorillo5d.com, EElcorillord.com, Elxorillord.com, E;corillord.com, Elcori;lord.com, Elcorllord.com, Elccorillord.com, Elcrillord.com, Elcoril.ord.com, Elcorilkord.com, Elcorillord.cmo, Ellcorillord.com, Eclorillord.com, Elcorillo4d.com, Rlcorillord.com, wwwelcorillord.com, Elclrillord.com, ElcorillordLcom, Dlcorillord.com, Elcorillofd.com, Elcorillore.com, Elcorillord;com, lcorillord.com, Elcorillorc.com, Elcorillord.c0m, Elcorillord.dom, Elcorillord.ocm."

Surely you have to see where many people would consider that either keyword stuffing, gibberish or typo spam.

[epi0Bauqu]

Thanks Matt. I was building a business off of these domains and realized that Google rankings were the biggest wildcard, and really didn't want any trouble. So I read the Webmaster guidelines closely and often and didn't think I was violating them.

However, I realize some were closer to the line and I should have focused on being less cookie-cutter and more useful in the domains that were really better (more farther along). I had always intended on coming back and working more on each, but wanted to get placeholders up quickly because it takes a while to get backlinks and indexed.

I guess I'm saying I had hoped I would have at least been contacted with a warning and what was found objectionable before just being totally blacklisted with no reason given. I would have also hoped that each site would have been addressed individually. If I had been contacted and you had said, hey, you need to remove these misspellings off of these sites, I would have done it immediately.

Here are some comments on the above though. Again, from my perspective these weren't violating the guidelines because the pages were useful from the user's perspective and there were no hidden tricks going on.

First off, there were actually many categories of sites, domains was just one of them. Others were sports stats, definitions, language, medical, and addresses. For each site I made, I was modelling it off of other sites that had gotten great Google rankings for years. I had hoped to eventually improve the UX on those sites and get similar rankings. For domains, I'm talking mainly about who.is and domaintools.com.

Each domain had a static site index, and that's what you linked to above in the screen shot. The extensive ones weren't really meant to be browsed, but just so search engines could find the pages (pre my knowledge of sitemaps). It's no different than any of the other static sitemaps, e.g. http://who.is/whois_index/index.php, and most of them looked better than the screenshot.

That one in particular came from the code for the streetsandzips site that was a big tag cloud. I was trying to find ways to make the static site better, and that was one of them. It looks better when the fonts are of different sizes :). I had intended for that site to make them different sizes based on the traffic numbers, so Google, Facebook, would be really big, etc. On the streetsandzips site the bigger cities are bigger.

In fact, I believe I evolved the sites so that those (site index) pages had noindex,follow on them such that they wouldn't come up on search results. I also added a search engine (Google custom search) on each page as well. I don't remember if I got to the tag cloud sizes for this particular domain at the time of blacklisting.

As for the misspellings, I did mess around with those, but not on all sites, and I believe at the time they were blacklisted that had been removed from most of the domains, if not all.

Common misspelling and typos as you know is a tool that people provide to those who buy domains. I built it for that purpose, and wanted to see how many people were actually searching for this stuff, so added it to some of the domains. Turns out, a lot of people do. I didn't just tack it on to the footer or cloak it or whatever; I put it in with a purpose that people ask for, e.g. common misspellings and typos.

Additionally from the users perspective, if they got to this page by typing in one of those misspellings, they were getting a big link to the official site at top and then more information about that site, e.g. siteadvisor rating, traffic, etc. So it was essentially functioning as one-click Did you mean x.

I'm happy to answer more questions about it. But it is pretty clear that it was still shoot first and ask questions later. No one ever contacted me about anything. I wasn't trying to hide anything from Google. It was all in my personal adsense account.

I can understand from a search engine perspective, banning sites. But given I already had a relationship with Google, I expected to be contacted. In fact, at one point I had a call with an Adsense guy from Google trying to help me better optimize my sites for Google! He looked at them and had no issues with them, so I thought I was fine.

Also, IIRC I submitted at least one re-inclusion request after being banned, and never heard a response back from that either. Before submitting that request I did a top to bottom review and tried to remove anything even close to the line, including misspellings I believe.

[alexblack]

From what I've seen Google doesn't contact people :) My guess is they also have a policy of not sharing reasons for getting blacklisted, to ensure they're not giving spammers an easy way to fix their website.

They claim they respond to all "Site reconsideration" requests. I had to file one once, they did respond, but with a very non-informative and unhelpful response.

[epi0Bauqu]

Yeah, in retrospect I should have taken it slower and not gotten as close to the line in the first place. It's totally my fault, and I'm not bitter. As you can tell from the OP, I've had a lot of failure, and I similarly learned from this one.

[Matt_Cutts]

The tricky part is that the math works out something along the lines of there being ~200,000,000 domains and there being ~20,000 Google employees. At a simplistic level that works out to 10,000 domains per Google employee. Which means that even if Google stopped doing everything else and everyone at Google spent all their time talking to webmasters, they'd each have to answer 10,000 peoples' questions about rankings, how to make their site, whether they have ranking issues, etc. That's oversimplifying somewhat because there's lots of parked domains, but not too much--you'd be surprised how many people want to talk about their parked domains and why they aren't ranked the way they want. My team is vastly smaller than the number of Google employees, of course. And our first order of business has to be worrying about what users see when they search; talking to webmasters is the secondary priority.

The net effect is that we haven't found a way to talk 1:1 with every webmaster, and I'm not sure whether that's possible. The story of webmaster communication for the last few years at Google has been trying to improve scalability of the info. The earliest Google webmaster communicator ("GoogleGuy") answered questions on a webmaster forum. In 2005 I started a blog, which has the advantage of permalinks for posts like http://www.mattcutts.com/blog/seo-mistakes-autogenerated-doo... . We tried doing live webmaster chats, but that would only reach 400-500 webmasters at a time.

The most scalable thing I've found so far is making videos. Here's a video that came out last month about the dangers of autogenerating pages for example: http://www.youtube.com/watch?v=A8bgpWtVHo4 . We're at almost 300 videos now, and we're getting closer to 3M total views on our webmaster video channel. The hope is that this additional guidance helps people self-identify what can cause issues to avoid or to correct them without needing to talk to Google.

The other big tool that has been helpful is http://google.com/webmasters/ . That provides tools to identify the common errors/mistakes that webmasters make (crawl errors, 404 pages, canonicalization, robots.txt issues, identifying hacked sites using the "Fetch as Googlebot" feature, etc.). That helps with many of the straightforward issues, but of course it doesn't solve the issue with "sheer number of webmasters who have ranking questions vs. number of Googlers." If anyone has suggestions on how to tackle communication with webmasters in a more scalable way, I'd appreciate feedback on how to do better on that.

[epi0Bauqu]

I think the best scalable thing that you could do would be to generate a lot more useful automated warnings via all registered channels. And then have a process you outline somewhere on timelines and how to correct. I think the biggest hassle from the user perspective is it all feels like a black hole and black box.

I understand the argument behind keeping it a black box, but it doesn't need to be as much of a blackhole. For example, in this case the following could have happened:

1) Site triggers some alarm for violating something.

2) Just those site(s) get strongly penalized.

3) Automatic emails go out in the message centers of Google Webmaster tools, analytics, adsense, and Gmail -- wherever the sites show up registered. In my case, it would have been all of the above.

4) The messages indicate the nature of the violation, that there is a penalty in effect.

5) There is a link to click on if you think you've corrected the errors.

6) If you click it, it auto-checks your site in y days and sends you another message that it passed or not.

7) If not corrected, it stays penalized or there are a series of penalties until full blacklisting.

That's all automated, i.e. scalable. I understand there are some tricky bits about how much to reveal about why things were penalized and what not, but I think those could be worked around usefully.

[dfabulich]

Matt wrote a blog post about this http://www.mattcutts.com/blog/notifying-webmasters-of-penalt... and published a YouTube video http://www.youtube.com/watch?v=hTI1TSmKmps

Google attempts to determine whether you deserve a warning; the goal is to notify honest folks, without notifying real "bad guy" spammers that they've been caught. Naturally, the algorithm gets it wrong sometimes... detecting wrongdoing is easier than detecting intent.

Were you blacklisted from Google Search or Google AdSense or both? Google AdSense's blacklist policy is totally separate from Google Search; Google AdSense's policy is to blacklist people on suspicion of wrong-doing (guilty unless proven innocent).

[codinghorror]

First, Matt Cutts is awesome. I don't think this can be disputed.

Second, we're doing our part to spread what we're learning about running a top 500 website (Stack Overflow) with the community, in the form of http://webmasters.stackexchange.com

Do we make mistakes? You bet we do. Just the other day I accidentally disallowed all questions on Stack Overflow from being spidered in robots.txt. That.. was .. not a good day.

[Morr]

The videos are great. Also, webmaster central has a ton of great info.

However, I interact with a lot of customers who seem put off by webmaster central. It seems to be a very outdated interface. I understand it's important to be clear and concise when explaining these issues. But if you look around the web 2.0 world at people providing similar information there's a harsh contrast.

Put simply, webmaster central is small text with a dark appearance and little or no graphics. In my experience and testing this harbors a mentality of "This is too complex". Users seem to encounter long wordy pages with no graphics and convinced themselves it's beyond them, before they begin to read.

Making a page lighter and throwing in a few visual aids goes a long way in curbing this issue, as well as making the information easier to understand and more fun to read.

It seems like a small thing, but it scales to become overwhelming when you consider that most people who encounter a page like this and dismiss it at a glance start looking for an email us link or a contact phone number.

This is my experience anyway. Perhaps your results may vary.

[Matt_Cutts]

Morr, thanks for the feedback--I'm talking to that team in 45 minutes, and I'll pass on the advice. The webmaster console has evolved a lot through the years, but I'd be the first to admit that it's a lot of info on a relatively small amount of pages. Lately the philosophy has moved more toward "Let's try to set up the tools to solve the most common questions or problems that come up." That could work better than a passive panel of information that doesn't tell you what to do about all the info you see.

The idea is still percolating, but I think it's got a lot of potential.