CPU may not be your bottleneck... Mailinator is handling up to 2 million emails an hour off a single server. Bandwidth though is an issue, 3TB a month. You pay for all those connections that spammers open.
Yes, I expect bandwidth to be the bottleneck. But it can be pretty cheap: serveraxis for example has unlimited bandwidth at 10mbps starting from $40 a month.