DietPi http://dietpi.net/
Minibian https://minibianpi.wordpress.com/
Both Raspbian-based
And, in just 80MB, Slitaz-ARM http://arm.slitaz.org/
Though I don't know if the latter has SSH enabled by default.