[prawn]

Is that the key though? There's a lot of stuff I'd be quite happy if they didn't index!

[chaostheory]

Imo Yes, because that’s the main difference between Google search and everyone else. Out index Google, and it’s possible to beat.

That said, I do like the feature of ignoring entire domains. Google used to allow that