They probably didn't even know that it was accessible via that domain. Their webserver responded to any request with the default site, and google decided to crawl ssl.nickjanetakis.com and found all the pdfs.
That's what I thought too but then I noticed that someone bothered to put "- Nick Janetakis" in the titles of those PDF pages (check the screenshot in the article).
I don't think that's exactly what was going on though, although perhaps somebody else can chime in.
I don't think the "- Nick Janetakis" is actually in the title of the PDF, rather google has appended it to the actual title (the end of which has been replaced with an ellipsis).
I think google can get this from either the title of a html page or from a og:site_name entry of a html page (I'm not 100% on all this). It's possible that google took these from the "actual" ssl.nickjanetakis.com and still remembers the og:site_name and applies it to the pdf files?
I think that may be something Google appends to its search results for some links?
I googled my own domain (site:flurdy.com) and it appends "- flurdy" to some of my static pages. But not for all, especially not for subdomain apps. So I am not 100% sure.
What's the advantage of that compared to using a bare IP? You still need a server to host your illicit content, so you're still exposed that way. Also, DMCA requests are sent to service providers, not to whoever owns the domain. The whole arrangement is probably worse than a bare IP because your site can be "taken down" by someone else with no warning.
At a school I use to attend, their firewall filtered "inappropriate" content (which is a fun story on it's own...). Was a poor system, and in theory, would have a loophole around it...
I guess a spammer would be able to use the domain ranking for their spam until Google detects that content has changed. Probably easier than promoting a new spam domain