Sendgrid's Event API (http://docs.sendgrid.com/documentation/api/event-api/) does something similar to this. Their mail servers act as a gateway to allow your app server to both send and recieve email over HTTP. I've used it at work as part of a not-yet-released feature of our product.
Real men write their own email servers ... kidding ;P
I think most people want the parse api. That is doing something when an email received. We were going to use that and the event api ourselves but I ended up writing my own custom email server with subethasmtp (in hindsight the lost opportunity cost was probably not worth it.)