Beware of super cheap VPS providers. A couple years ago I got burned by a provider called VBOX that was highly recommended on lowendbox.com. At first the service was ok, but a few months later my server and VBOX business were completely unresponsive.
Sure you'll save some money, but you might also waste hours of your time setting up a server that could one day disappear into thin air.
So long as you have deployment automated, and frequent backups, that shouldn't be a huge issue. One of the many advantages of not having to manually deploy a new box is being able to rapidly get off a bad provider.
Debian minimal, with NGINX or some other light HTTP is easy on even 64 megs. On 128, you can do stuff like PHP and you can run whatever efficient web stuff you like. The web is light, if you're efficient about it.
As an aside, one of my friends has a 100 meg free VPS from one of the terrible free vps providers. He has managed to squeeze a Minecraft server on, and that can hold about 2 or 3 players! (apart from the terrible network, cpu and uptime)
It isn't (I have one, a whole $15/year due to some ridiculous sale some months ago). Consider that it's userspace only, so kernel doesn't count. I'm running a (light-duty) nodejs app, nginx to proxy it, tmux, and two irssi instances on mine right now, with memory to spare. Light PHP works fine too (haven't tried anything heavier).
A database server or similar wouldn't work, of course.
If you pay $2/month, you're a sucker; they've got lots of coupons (like the seemingly permanent 25 % discount on their website. If you look a bit further, you'll find even better discounts).
I use INIZ[1] 1GB for only $25 a year from a lowendbox special[2]. I don't use it for much so I can't speak with quality/uptime/anything beyond testing/experimentation.
1GB-YEARLY
3 vCores
1GB RAM
512MB vSwap
50GB RAID10 Disk
1TB Outbound BW
1 IPv4 and 1 IPv6
OpenVZ/SolusVM