I have a Linux VPS I use everyday, mostly just privately. It's very handy as a way to get around the corporate network blocked sites, and I can use it if I feel the Wifi network I'm on is sketchy to try and encrypt my traffic. Mine is only 3 bucks a month and that's easily worth it to me. Lowendbox.com has good deals pop up all the time. It's no speed demon, but it works for all I do with it.
Lowendbox.com is a great way to get cheap VPS service, and anything <$10/mo is hard to complain too harshly about.. but my caveat to using companies from there is to make sure you have decent backups, handle email/DNS somewhere other than your VPS, and try not to care too much about 100% uptime :)