[tlrobinson]

I'm still not quite sure where Mongrel2 fits in the stack. Is the idea to just take care of the nitty gritty details of writing an asynchronous HTTP server and expose it to app servers via ZeroMQ? Like an asynchronous version of FastCGI?

If I already have an app server that has a good asynchronous HTTP server built-in (Tornado, Node.js, etc) is there any reason I'd want to use Mongrel2?

From the documentation:

"Other web servers will let you talk to any language as a backend, but they insist on using HTTP proxying or FastCGI, which is not friendly to asynchronous operations."

Maybe I don't understand why HTTP proxying isn't async friendly? Node.js does it quite well.

Not trying to knock Mongrel2, I'm just trying to understand better.

Zed if you're listening, some example deployment scenarios would be helpful.

[zedshaw]

Good questions. It fits at about the sample place you'd put nginx or apache, with it in front of your other language application stacks. Mongrel2's protocol also tends to remove the need for any "middleware" like WSGI or Rack since it's protocol is already similar to what those do (minus the insane chaining and wrapping they do).

The reason I say "HTTP isn't async" is that, while it can be async over a single socket, you can't really do N:M responses with it. For example, you can't have nginx send a request to one Python backend, and then have 6 other backends respond directly to the client before closing the connection in a 7th. Mongrel2 does this by sending 0MQ messages to one target, but allowing any handler to remotely "write to the browser socket" with any number of 0MQ response.

The closest you can get with HTTP is having specially written proxies that know how to do individual HTTP requests to each backend, and then take those responses and use chunked-encoding to build the final response. I believe Amazon does this fairly effectively for their applications (or used to).

So, I think the confusion on the term "async" is that you're thinking "async socket IO via callbacks in Node" vs. "10 processes asynchronously responding to one browser".

That answer it?

[tlrobinson]

That helps, but I guess it begs the question of why you would want "10 processes asynchronously responding to one browser"? And what does that really mean?

It doesn't sound particularly useful for traditional HTML responses (unless you had some kind of simple template system to aggregate them into a coherent response), but I could see it being interesting for WebSocket applications, especially multiplexing multiple backend services over a single connection. For example, on Facebook's chat and news feed services multiplexed over one connection. Is that a typical use case?

It would be really awesome if Mongrel2 could support something like Socket.IO or Orbited, which provide a WebSocket-like API to the browser, but automatically negotiate another transport if WebSocket isn't available.

Maybe it already does, I don't know. I'd just like to be able to use something like WebSockets without worrying about the capabilities of the browser or backends (Socket.IO is for Node.js and a few other platforms. Orbited seems like a dead project, but I believe it used the STOMP protocol).

[zedshaw]

> That helps, but I guess it begs the question of why you would want "10 processes asynchronously responding to one browser"? And what does that really mean?

Two use cases: A background task that involves a few systems, but you don't want to block the main app stack. Simplest case is say image crunching on an upload. More complex case is what you'd do with something like Gearman.

Second: You can carve up your pages into each piece and plug the pieces together. This would require a bit of coordination, but you could have a backend for headers, one for content, side bars, footer, etc. and they all respond when they get results directly to mongrel2's socket with a chunked-encoding.

> It would be really awesome if Mongrel2 could support something like Socket.IO.

It already supports Flash sockets and long polling, so Socket.IO should just work. Haven't tried it though. Check out the docs and look at the chat example in examples/chat/ for a quick (slightly old) demo that's just using jssocket.