[zacharydanger]

This is probably beyond the scope of such a thread, but configuration automation via Chef or Puppet is where you'd start.

[eli]

You really think that's the best place to start?

I would begin with a figuring out a more secure, desired configuration before trying to automate it. (Especially given, IMHO, the very steep learning curve for Chef & Puppet)