[joefkelley]

I used to work at a big data consulting company and dealt with hadoop clusters at a bunch of different companies. What you described was absolutely the norm. The entire cluster closed to the outside world, except for one gateway machine that allows ssh access, and anything within the cluster is totally open. Sometimes some web services were open to the company VPN.

Kerberizing is a pain but not usually needed. You're correct that AWS firewall rules are very easy.

What you're seeing in this article is the exception, people doing it totally wrong.