[blossoms]

Unrelated but it looks like www.aol.com's robots.txt is served as text/html http://www.aol.com/robots.txt

Is this a common mistake?

[saltysugar]

Yes, from the server side's mishandling of TXT extension. Probably the server put the MIME type in the HTTP header as "HTML" instead of TXT, and the browser renders the page as such.

[pjscott]

From the headers:

    Content-Type: text/html;charset=UTF-8

It also tries to set no less than four cookies.

[mkonecny]

It just issues a "HTTP/1.1 302 Moved Temporarily" directed to their homepage. Requesting an invalid file such as "robots.txtsdfa32r523" has the same effect, so they probably don't have a robots file at all.

[blossoms]

http://i.imgur.com/Jmjf3y2.png

[judk]

Huh? No, it is a regular robots.txt file

[tonyedgecombe]

It redirects requests from the UK.