If I published a list of all name and addresses, that's still different than "here is harywikle's full name and address". I imagine you wouldn't be too pleased?
That's the issue I take with the "phonebook" defense. It justifies doxing people by collecting and connecting publicly available information online. All the information is out there, it's all on a phone book, your email was published online, and so on, but the end result is clearly bad so something in the process should be handled more carefully.