GoKit does that (http://gokit.io) – I heard about it on the changeling podcast [1]. Really cool set of scripts. Haven't used it personally though. There's also a changelog episode about Caddy [2], although I'm sure some of that will be outdated starting today.