I think it's only ghosting if you break off a conversation, not if you never start one. If I send you Viagra spam and you don't reply that's clearly not ghosting.
I think "ghosting" is also used when one person breaks off a conversation, but the other person continues the conversation despite the conversation being broken off. At that point, an obnoxiously persistent person may conclude that that are being ghosted, rather than noticing that the conversation was over.