[nkoren]

I'm not really a Python programmer, but a hacker who occasionally has cause to pick up Python scripts and do stuff with them. Perhaps I've been unlucky, but every time I've done this, it's turned into a profoundly frustrating exercise. There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my environment is set up for. I've frequently run across scripts with dependencies that somehow only execute in mutually incompatible versions of Python, which always makes for an exceedingly aggravating day of programming.

As much as people love to bash PHP -- and I agree that it's pretty awful as a language -- its standard library is so comprehensive, backwards-compatible, and superbly-documented that I have never had a comparably aggravating experience with it. The same is true of Javascript: a language with warts, but whenever I try something, it Just Works.

Like I say, perhaps I've just been unlucky, but my distinct impression of Python has been that it's a beautiful language surrounded by a particularly problematic ecosystem of incompatible libraries and sparse documentation. I suspect that the Python community would benefit from paying less attention to the purity of the language, and a lot more attention to the quality of everything surrounding it.

[espeed]

There have always been dependencies outside the standard library, and those have had dependencies -- which more often than not, are incompatible with whatever version of Python my environment is set up for.

Create a virtual environment for the Python version you want to use, and install the package with pip (pip will install all the dependencies in your project's local environment)...

  $ mkdir myproj
  $ cd myproj
  $ virtualenv --python=python2.7 env
  $ source env/bin/activate
  (env)$ pip install somepackage

[mas1n]

Or you can install virtualenvwrapper (http://www.doughellmann.com/projects/virtualenvwrapper/) and after a little configuration...

  $ mkvirtualenv env1
  (env1)$ pip install somepackage

Creating self containing python projects has never been so easy!

[nkoren]

Yep, I learned this trick on my first go-around. The problem has occurred when I've tried to run scripts with dependencies which somehow are only available in mutually incompatible versions of Python. I'm not sure how this is possible (it's baffling -- it shouldn't be possible), but it's happened three out of four times that I've tried to use a Python script of any real consequence. Usually after half a day of futilely trying to find an environment which will actually accommodate all dependencies, I end up having to port everything to whatever version of Python appears to be the most common denominator. In all fairness to Python, this is relatively easy to do (except in one case, when I had to hire a Python-expert friend to do it), but it still means that what should've been a five-minute affair (less the dependency hell) turns into a full-day affair.

Like I say, I've probably just been unlucky. But at this point it's given me a pretty serious aversion to Python. Will probably have to get over that someday, I suppose.

[anthonyb]

Which packages? I'm curious, because unless you're trying to work in Python 2 and 3 at the same time, I can't think of any packages which don't support 2.5, 2.6 and 2.7. And I've been programming in Python about 10 years or so. Unless you're going super-bleeding-edge, it shouldn't be a problem.

[anthonyb]

I hate PHP's standard library - I find it to be weird and inconsistent, arguments are in a random order, etc. and Python to be not so bad. I suspect this is because you have a lot of PHP experience, and I have a lot of Python experience :)

Virtualenv and PIP go a long way towards fixing these problems in Python, similar to how PEAR and CPAN work with PHP and Perl.

[slurgfest]

Having used Python in anger for some years I have never encountered anything like this (scripts with mutually incompatible dependencies, really?) It sounds like you ran into some specific package which was poorly made or had a bad release, or perhaps just a package which should have had its dependencies pegged at specific versions?

I'm completely lost as to why you think this is a problem with Python or its standard library. (Except that this seems like an opportune venue to bash a perceived competitor to your favorites)

It was years before I even saw any need to use virtualenv, (and that was because of misbehaving packages from Google and a desire never to modify PYTHONPATH again).

About documentation, again I feel strongly that you must have used some specifically bad package and again I can't say how I see that to be an inherent flaw with Python or its standard library.

"Pure" is one of the last things I'd call Python. For example, imperative constructs are jumbled with OO constructs and functional constructs.

[fuzzix]

Most frustrations I've had with Perl/Ruby/Python have been down to inconsistent, incomplete or out-of-date packaging by the software distribution.

I've found I can mitigate most of this by using perlbrew/cpanminus, pythonbrew/pip and rvm/gem.

This adds complications when it comes to deployment - you've added additional maintenance dependencies to your servers if you don't stick with what the package manager provides... Nothing insurmountable, but keeping an eye on updates (would usually use email to notify on dated versions) and maybe having your CI system always use the latest versions of everything are possible methods. I'd like to hear better ideas, to be honest.

My background is both sysadmin and development, I think there should be no reason we can't make this all simpler for everyone.

[slurgfest]

Yes, I agree completely.

At least in Python, the problem occurs because distros make several mistakes which build on each other. (A) insist on generating their own packages for modules (and typically, taking forever to update them). (B) not providing any kind of isolation. (C) building on top of these non-isolated modules.

What I do these days is leave the system Python 100% for the use of the distro (at most, dependencies for minor command line scripts which don't have import relationships with code I'm working on). Development occurs inside virtualenvs, production apps run inside virtualenvs. I use pip to install, remove, pin versions.

Dependency versions are really part of the app; the right versions should be pip installable inside a virtualenv, and this is just something which should be done automatically on a deploy. I don't mean you need to include the whole code for your dependencies in your project, but pinning versions is important to not having to put out fires.

You really don't want any versions to advance automatically until you have had a chance at least to run unit tests - if things are working then pin the new version. Of course, if you are using a library which never suffers regressions or API changes then you don't need to do this.

In short, the dependency list and versions should be included with the project and managed by the developers so that deploys are really a matter of creating a virtualenv (with --no-site-packages) and running pip -r requirements.txt to install the right stuff.

[heretohelp]

I think you've really just been unlucky.

I've had the same experience before, but chiefly with Ruby.

Interestingly, my problems with this in both Python and Ruby have evaporated once I got in the habit of using virtualenv/rb-env/rvm for my development environment.

Python is about the cleanest/nicest experience I have in any language, for the record. Only language that comes close is Clojure.

Leiningen is...legendary.

[xaa]

I would argue that Python's approach is FAR better than Clojure/Leiningen's "no batteries included" approach.

Suppose you want to do a very common task like parse some XML. In Clojure, the workflow is:

  1. Go to Github or Clojars, find the latest version number  of clojure.data.xml

  2. Add this version number to your project.clj

  3. Lein deps and restart the repl

  4. Re-acquire whatever REPL data you had

In Python, it's:

  1. import xml.{sax,dom,etree}

And, paradoxically, the availability of all these different versions of libraries in Clojure leads to MORE conflicts between libraries than would otherwise be the case, not less. In Python, you may not agree that, say, the "os" or "subprocess" modules are optimal -- but by golly, they're consistent.

[dpritchett]

Thanks to pip I often don't even bother with the Python stdlib for crusty things like one-off web scrapes or XML parsing. Here's a recent example where I wanted to read some attributes out of some remote XML and did it with requests and PyQuery rather than urllib and xml:

    _domains_text = requests.get(API_URL + "/domainlist.xml").content
    _domains_db = pyquery.PyQuery(_domains_text)
    
    DOMAINS = [d.values()[0] for d in _domains_db('domain')]

[RegEx]

I like to set

    jQuery = pyquery.PyQuery(someHTMLDocumentString)

So I can use jQuery like I'm used to. At this point, you can do

    links = jQuery('a')

or whatever.

[leephillips]

I've had the same experience before, but chiefly with Ruby.

Tell me about it: http://lee-phillips.org/badruby/

[jamesgeck0]

Is it reasonable to expect packages to maintain compatibility with a version of Ruby from 2004? I don't know many Python libraries that still work with Python 2.2. Not sure about Perl 5.8, but I couldn't even find a link to download a binary.

[leephillips]

Is it reasonable to expect packages to maintain compatibility with a version of Ruby from 2004?

That in itself would not be reasonable. But it seems as if Ruby's own libraries broke going from 1.8.1. -> 1.8.7. Regardless of the number of years involved, that's kind of unexpected. But I'm not familiar with the Ruby world, and maybe that change in version number is considered major.

The whole experience left me with reduced confidence in Ruby stuff and I still avoid it and programs that use it. The comments below by people who know more about the Ruby ecosystem than I do don't give me any reason to change.

[angelbob]

(Disclaimer: I am a professional Ruby developer)

The problem is that the various packaged versions of Ruby are a shambles. RVM has a number of significant problems and difficulties (rbenv is both better and worse).

Right now getting a good, modern Ruby version on a standard modern computer is a big pain in the ass.

As long as that stays true, we will be dinged for not supporting old (but common) versions.

Years from now, when everybody has good 1.9 compatibility, things may be better. At least, if 2.0 doesn't have the same problems...

[jes5199]

the problem is that it's much, much harder to upgrade ruby from 1.8.7 to 1.9.3 than it is to upgrade perl from 5.8 to 5.14

So many of the language features have changed in ruby programmer will have to edit every file in her project, and upgrade every single dependency, just to get her application running on a new ruby interpreter. Add that to the culture of "let's use as much 3rd party code as possible!" and the library writers emulating the core developers and changing their APIs all the time ... the process of upgrading an interpreter converges on "rewrite the entire application"

Which is what we do. And some percentage of those rewrites are in languages that don't have this problem.

[slurgfest]

If you know what you are doing then you know what version of the interpreter you want and have the ability to install it. Once upon a time I wrestled hard to compile tarballs, these days it is really rare that I have to do more than ./configure; make; make install in the worst case.

[darklajid]

What is Leinigen offering over Maven? Isn't it based on the same stuff?

That said: Leiningen & npm _are_ really nice solutions for me.

[chaostheory]

I think the problem is because when you used it the Python community probably still hadn't decided on a one true dependency system like Gems / rvm for Ruby, and Maven for Java. It's been about four years since I've heavily used Python, but hopefully this has been fixed by now.

I don't have this problem on either Ruby or Java.

[slurgfest]

Nope, pip and virtualenv are pretty much it. Use 'em