[justin_vanw]

"What WSGI should have been."

Well, not at all. Pump is what werkzeug, webob, and other more friendly wrappers on WSGI already are. Basically pointless duplication of work, without understanding why WSGI can't be this simple.

One basic reason that WSGI can't be as simple as just returning a dictionary is that you don't necessarily want the entire body of your response pre-computed before starting to return data to the client. What about long running connections? What if you want to return the head of the response immediately, so the client can start pulling css and js while you compute the body of the response? What if you want to do chunked encoding to support long polling connections, or responses where you don't know the response size beforehand?

Pump is basically doing what lots of other things already do, except without quite understanding HTTP quite as well.

[pyninja]

Thanks for the feedback. I'm not willing to accept that WSGI couldn't be this simple. Pump's specification is modeled on Ring for Clojure (https://github.com/mmcgrana/ring), and I'm sure there's a way Ring gets around the issues you mention. I'd never had to do any of those things before but I'll look into it now.

[justin_vanw]

Werkzeug and WebOb already do what you are trying to do, and they do it in a completely fantastic way. I'm not saying that to discourage you from learning and contributing, but what do those two libraries do wrong that you are hoping to improve on?

In the meantime, don't bash WSGI until you understand why it is the way it is.

[pyninja]

Werkzeug and WebOb are not what I'm trying to do.

I'm reposting a reply I made earlier to irahul:

Pump aims to replace WSGI entirely. That is, I believe it does a better job of what WSGI was intended to do.

I understand your point that web developers don't necessarily work with WSGI on a day-to-day basis. But if you look at the Ruby web community, Rack middlewares are much more prevalent than WSGI middlewares. Application developers (as opposed to framework developers) often add functionality as a Rack middleware, so that it can be reused in different applications, even using different frameworks. Why isn't that happening with Python as much? In the Python world, instead of writing even simple middlewares to for basic functionality like https://github.com/adeel/pump/blob/master/pump/middleware/pa... or https://github.com/adeel/pump/blob/master/pump/middleware/co..., every framework ends up reimplementing it. I believe this is because the WSGI API is ugly and not as easy to understand as it could be (just look at the average WSGI middleware).

[justin_vanw]

What middleware do you think are missing? In general, stuffing application logic into middleware leads to problems, so I don't think your premise is sound.

http://dirtsimple.org/2007/02/wsgi-middleware-considered-har...

I think your approach is harmful and will put developers who use this library in a very bad situation. You are putting a simplifying abstraction on top of HTTP. As with all simplifications, things which do fit perfectly in your tool become very easy, and things that don't fit perfectly become completely impossible.

Finally, I stand by my argument. Werkzeug and WebOb let you put the exact kind of simple interface on top of WSGI that you are attempting to, but with the benefits of not restricting you to a subset of HTTP, and interoperability with awesome tools like mod_wsgi.

[clarkevans]

If you are really seeking to replace WSGI, then I'd encourage you do a bit more research, write a specification, get at least one more implementation, and then submit it as a PEP.

Documenting at this level is challenging & rewarding. The hard part about specification work, beyond explaining the design and providing adequate justification, is getting consensus. By doing so, you'll learn quite a bit, open yourself up for critique, and in the end perhaps provide a credible alternative.

[pyninja]

I'd like to do that, but I'm afraid of how much time and effort it would take. I really just wrote Pump/Picasso to use at our new startup, Beagle (http://beagleapp.com). I was expecting roughly this kind of response from the Python community, but who knows, maybe someone will use it.

[mkramlich]

I'd advise you to spend more of your time on building what's the core of your new business, because there are tons of higher priority things you'll have to accomplish. Web frameworks, even "lightweight" ones, in Python, is/was not an unsolved problem. Lots to choose from. Pick and go.

</speaking-from-experience>

[jessedhillon]

At the risk of sounding harsh: it's an incredible failure of time and project management for you to have written your own Python web framework as part of launching a startup.

[Confusion]

I think the pertinent question here is: what do you think 'WSGI was intended to do.'? From the responses here, it seems your answer to that question is a distinct subset of what WSGI actually intends to do. Which is why Pump offers only a subset of the abilities of, for instance, Werkzeug.

[mkramlich]

I'm pretty sure the plural of middleware is middleware.

[sigil]

As a special case, how about allowing the response body to be an iterable, and writing whatever blocks of data it produces back to the client?

You should also allow multiple headers with the same field-name, since that's in the spec.

I like the idea of Pump, and am tired of frameworks protecting me from HTTP.

EDIT: Looking at the WebOb code. It does have quite a few conveniences for working with HTTP messages. I'm not sure if copying bodies into temp files in order to make them seekable is a "completely fantastic way" of doing things, but if I were you I'd definitely read through WebOb to see what kind of problems you might be up against.

[justin_vanw]

WSGI specifically specifies the response body as an iterable. Wheel, prepare to be reinvented.

[sergeys]

WebOb maintainer here. What's your alternative to copying body to a temp file if you need to make it seekable?

[masklinn]

> I like the idea of Pump, and am tired of frameworks protecting me from HTTP.

if you don't like being "protected from http" why not write raw wsgi applications?

[sigil]

How is a gateway that reads, buffers, and parses HTTP headers into an environment object [1] before turning it over to your "raw wsgi application" not protecting it from HTTP?

WSGI protects you from HTTP. CGI protects you from HTTP. mod_python and mod_perl protect you from HTTP. If you're unable to read and parse the complete HTTP request yourself -- perhaps incrementally, there's an idea -- you're protected from HTTP. Something is imposing policy like how many headers to accept, what the longest header should be, how to fold multiple headers with the same field-name, that it's okay to consume memory buffering all the headers, and so on.

In my ideal world, a web app server has access to the full HTTP request stream, calls an incremental HTTP parser [2] [3], and does whatever it wants along the way. If the typical use case is to accumulate a full request object and call a handler, fine, that can be made convenient. But the web app gets to decide.

Perhaps my issue is not with frameworks (in the sense of Django, Ruby, etc), but with web servers. Except, I view the infrastructure for hosting a web app inside a web server as yet another framework. The common use case is optimized for at the expense of the less common use cases, which become more painful than they should be. Or sometimes outright impossible.

TL;DR -- Libraries over frameworks. In Soviet Framework Russia, you don't call code...code call YOU.

[1] http://www.python.org/dev/peps/pep-0333/#environ-variables

[2] https://github.com/ry/http-parser

[3] https://github.com/mongrel/mongrel/tree/master/ext/http11

[sergeys]

"Libraries over frameworks. In Soviet Framework Russia, you don't call code...code call YOU." -- exactly! Very few people get this, which is a shame.

[voidfiles]

Doing something simple is fine, but what I think other people are saying is that pump is reductively simple.