My concern about truly self-hosting this would be my sender reputation - you might find you end up in more junk mail boxes compared to using a service like SES.
Thanks man. Sender reputation is something I worry about. But half my clients get the email I send them using a simple phone script already. So I'm hopeful that is a good indicator.