Digital Ocean, Linode, Rackspace, Amazon EC2, Google Compute Engine
Digital Ocean lets you provision the lowest spec servers of all those, for $5/mo.
edit: if you just want Wordpress, check out https://aws.amazon.com/marketplace/ although I've never tried that service. They give you a VPS with various apps pre-installed, e.g., https://aws.amazon.com/marketplace/search/results/?page=1&se...