[postfuturist]

I'm the author. I've never worked at Microsoft (where did you get that?). I'm a Linux guy.

The "lazy people" you refer to is 98% of the Python community.

Python 3 didn't "gain" static type checking. You can (and I do) run pylint against Python 2 code which is just static analysis, not full type checking. The situation here has not changed between Python 2 and 3.

Python is glue code. You write a small piece of Python to tie together external libraries. Moving a smallish piece of Python code from Python 2 to Python 3 often means porting several 3rd party libraries. If you were to move to Ruby, all you'd need to rewrite is your own code, as there are likely already 3rd party libraries which need no porting.

[pdonis]

> Python is glue code.

Um, no. Major applications are written in Python. It looks to me like the problem is not Python but your perception of it.

[dagw]

From personal observation I'd say that for every person writing major applications in python you have at least 20 using it for glue code. And having written a major application in python, I'm not really convinced it's a good language for it.

[pdonis]

> From personal observation

Which is not a good metric. Have there been any actual surveys or studies of Python programmers to see what the distribution is?

> having written a major application in python, I'm not really convinced it's a good language for it.

Out of curiosity, what was the major application?

[dagw]

Nope no metrics, I doubt very much they exist. But if I can pile on a second personal observation, a significant proportion of people programming python aren't python programmers. They'er engineers, artists, scientists, analysts and statisticians whom are using python to do their job. So even if you somehow did manage to survey a statistically significant number of python programmers, you'd still be missing a large swath of the python community.

As to the application, it was a largely in house tool for an engineering and company I used to work for.

[pdonis]

> even if you somehow did manage to survey a statistically significant number of python programmers, you'd still be missing a large swath of the python community.

Depends on how the survey is done. If those engineers, artists, scientists, etc. are using Python, they're using Python, and there are ways to survey that. For one thing, you could look for users of the major Python frameworks listed in the Wikipedia article I linked to elsewhere in this subthread, like NumPy or SciPy.

If you want to claim that, for example, a scientist using SciPy isn't a real "python programmer" because he is only writing "glue code", I've already answered that in another subthread. Application logic is not "glue code". The scientist's application logic is in Python, so he's programming in Python, even if his job title doesn't say "Python programmer".

[coldtea]

>Which is not a good metric. Have there been any actual surveys or studies of Python programmers to see what the distribution is?

So where are your metrics?

[pdonis]

http://en.wikipedia.org/wiki/List_of_Python_software

This doesn't show how many programmers work on "glue code" vs. "major applications" (which is why I asked if anyone has data on that), but it certainly shows that there is a lot of Python code out there that is not just "glue code". Bear in mind that it's not just the particular packages listed on that page, since many of them are frameworks; there are also plenty of "major applications" written in those frameworks.

(I am counting major websites using a web framework, major numerical applications using NumPy or SciPy, etc., as "major applications", btw, not "glue code". I suspect on re-reading this subthread that the poster I was originally responding to might differ with me on that point, but application logic is not "glue code" in my view.)

[coldtea]

>but it certainly shows that there is a lot of Python code out there that is not just "glue code"

Well, of course, this list would have only NON glue code projects!

Glue code stuff is used internally, in companies etc. It's not (usually) something that you put on GitHub or open source.

[higherpurpose]

In every Python book I read they were recommending me to use Python 2.7 instead of 3.0. Shouldn't it be the role of educators to promote the latest version out there and work with it, so that by the time the new students get into a working environment they are already on the same page with what is required?

[maxerickson]

Prior to the release of 3.2 (Feb. 2011, 2.5 years after the initial 3.0), that was clearly reasonable advice. The io subsystem in 3.0 was quite slow, making it uncompetitive with 2.6, and while 3.1 fixed that, there were still too many missing libraries (and similar issues).

I wonder what would have happened if the software that was called 3.0 had been released as "Python 3 Preview Release" or something. I guess the thinking was that would have held people back even more from trying it/porting stuff, but maybe scaling back (end user) expectations for those first couple of versions would have been the better path.

[dragonwriter]

> In every Python book I read they were recommending me to use Python 2.7 instead of 3.0.

Well, for one thing, Python 2.7 is a newer release than Python 3.0 (2.7 was released after 3.1.)

However, while there may have been good reasons to choose 2.7 over 3.0 (or even 3.1), there aren't nearly as many reasons to choose 2.7 over 3.4.

[piokuc]

> Shouldn't it be the role of educators to promote the latest version out there

I think the main role of authors of books on programming is to help their readers to become more effective in solving their programming tasks. So they do, if they're wise and honest.

[pdonis]

> Python 2.7 instead of 3.0.

Maybe you need to read more up to date books. The latest release is Python 3.4, and even most Linux distros are at least at Python 3.2.

[claystu]

According to Distrowatch, out of the top twelve (12) distros, all but #8--Arch--use Python2.

The other 11 in order of popularity are #1 Mint, #2 Ubuntu, #3 Debian, #4 Mageia, #5 Fedora, #6 openSUSE, #7 elementary OS, #9 Zorin OS, #10 Puppy Linux, #11 PCLinuxOS, and #12 CentOS.

Most Linux distros use Python 2.

[chc]

That's not quite a complete picture. Those distros include Python 2 as the default "python" executable, but many include Python 3 as python3, and even more have standard python3 packages that just aren't installed by default.

[unethical_ban]

At work, I am writing tools in Python 2.6.6 specifically because the latest version of RHEL comes with only that, and I cannot get aftermarket versions of tools onto our servers. Is that the fault of Python? No, but I'm sure I'm not the only person working at a stuffy organization with crappy open source policy.

[rdtsc]

So do I (in some cases even stuck on Python 2.4) [RHEL 5]. But at the same time (re: Python 2.6) I never really thought "oh I really need the latest 3.4 or my project will fail". It doesn't offer dramatic speed improvement, concurrency, some nice built-in web framework or anything.

I would take 3.4 if someone else spent the time testing and validating my code against but I just don't have the time to invest because I just don't see an upside to it at the moment.

[rdtsc]

> and even most Linux distros are at least at Python 3.2

Why are you spreading lies? You are hurting your cause (advocating Python 3).

Let's take a look at latest Ubuntu:

   $ cat /etc/issue
   Ubuntu 14.04 LTS \n \l

   $ python --version
   Python 2.7.6

How about the popular server distro CentOS 6?

  $ cat /etc/issue
   CentOS release 6.5 (Final)
   Kernel \r on an \m

   $ python --version
   Python 2.6.6

What exactly are these "most" Linux distros everyone is using that I haven't heard of?

[pdonis]

> Why are you spreading lies?

Why are you assuming the worst possible intent on my part, instead of asking what I meant?

I meant the distros have Python 3 available, not that they make it the default that /usr/bin/python points to. Ubuntu has had Python 3 available for quite a number of releases; I run 12.04 which has Python 3.2 (and I've had it installed since I installed Ubuntu). According to the release notes[1], Ubuntu 14.04 makes Python 3.4 available, and work is ongoing to make it the default Python for Ubuntu. They also advise porting to Python 3.

It looks like CentOS makes Python 3.2 available, but I can't be sure from their online documentation.

[1] https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes

[rdtsc]

> most Linux distros are at least at Python 3.2.

I don't know how else to interpret that besides "the base version of python in those latest distros is at least 3.2"

Shipping it as an "alternative" just splits the library world into to. So that is not a distro being "at" a python version.

> It looks like CentOS makes Python 3.2 available,

It does. Do all the python-* libraries work with it. Or do I have to install those separately?

[pdonis]

> I don't know how else to interpret that besides "the base version of python in those latest distros is at least 3.2"

Then you could have said that, or asked if there was another way to interpret it, instead of starting with "why are you spreading lies".

> It does. Do all the python- libraries work with it. Or do I have to install those separately?*

If CentOS works like Ubuntu, then there will be python3-* libraries for use with Python 3.

[chc]

Many distros include both Python 2 and Python 3 support. I think pdonis is saying that distros that support Python 3 are not stuck on 3.0. That is, their Python 3 packages are 3.2 or higher, so comparing 2.7 to 3.0 is unreasonable, since it's not a choice anyone has to make. Which seems to be true. For example, the relatively conservative Debian is currently on Python 3.2, with Python 3.3 in the next version. Ubuntu's python3 is currently 3.4. Fedora appears to be 3.2. Arch has Python 3.4, I think.

So what pdonis was saying is, as far as I can tell, true.

[rdtsc]

They do. But python3 is not the default python.

Download a python script run it and it will run on Python 2.7.6.

Yes that OP meant to say that because he clarified it later. But I don't see how his original comment of:

> most Linux distros are at least at Python 3.2

implies that. To me that says 3.2+ ships as the default version of Python.

[pdonis]

> python3 is not the default python.

The post I was responding to talked about books on Python for students, presumably students studying programming. Python 3 doesn't need to be the default python for students studying programming to use it; it just needs to be available. If the post had been talking about, say, sysadmins, that would be a different story.

[rat87]

I think it's recommended for distros to modify the shebang to /usr/bin/env python3 for script that are supposed to run under python 3, although this does pose a problem for scripts that want to run under both without modification. If the distro ships with python3 and most of the system scripts run under python3 I'd say its the default.Ubuntu for one is planning on not shipping a python2 or /usr/bin/python binary by default(although it will still be in repos) eventually.

[rdtsc]

I think 3.0 was meant to be 3.x not 3.0 specifically.

[Signez]

You should try to run

    $ python3 --version

...instead.

[rdtsc]

Will the programs or scripts I download know to to that?

[jamesgeck0]

Any script worth it's salt that depends on a newer feature of Python 3 can just check `sys.version_info`. This was a thing one had to consider back when new versions of Python 2.x were coming out, too.

[fleitz]

Why? Languages are tools used for communications, if there is no one to talk to there is little point in learning a language.

English is a much better universal language than esperanto ever will be for this very reason.

[illumen]

There was a link to a Microsoft person in the top right of your page. I mistook it for the author link. Edited post to remove that.

You are right that the same tools work for python 2. However, I mentioned tools there which allow python to do type checking, and type inference because they can be done now. See here on how to use them: http://renesd.blogspot.de/2014/05/statically-checking-python...

You're right in that case where you use several libraries that aren't ported. However, there's 4799 packages registered on pypi as supporting python 3, and most of the popular ones are ported. Considering that most libraries are ported, porting small scripts should be easy most of the time. I don't know if you actually ported any code, but it's often trivial.

[zerd]

4799 out of 44198 are python3? just a few more to go.

[andybak]

Not sure if you're just trolling, but the majority of the rest are probably close to being abandonware.

[vram22]

>Python is glue code. You write a small piece of Python to tie together external libraries.

Glue code is only one of the things that Python can be and is used for. Python is used for small, medium and large [2] applications as well.

Do all of the following look like glue code?

Recent marketing brochure from the PSF (Python Software Foundation) - (it's a PDF):

http://brochure.getpython.info/media/releases/psf-python-bro...

Python success stories:

https://www.python.org/about/success/

Google's web front-end (many of their URLs even end in .py)), Dropbox clients (wxPython), YouTube [3], Disqus are some apps / sites written in Python.

[1] See the paragraphs about YouTube near the end of this post (by me, but much before this current thread):

http://jugad2.blogspot.in/2013/03/youtube-dl-yourube-downloa...

[2], [3] The PyCon talk below is by one of the original engineers at YouTube.

https://us.pycon.org/2012/schedule/presentation/128/

The Wikipedia article below says YouTube is one of the largest web sites in the world. A lot of YouTube is written in Python.

http://en.wikipedia.org/wiki/YouTube

[akerl_]

Do you have examples for these critical 3rd party libraries that haven't yet been ported?

As I noted elsewhere in this thread, the list gets shorter every day:

http://python3wos.appspot.com/

[bitL]

If you are running multiple automated eCommerce companies that have their core system written in Python 2 like I do, you'd find that boto/mws and python-excel aren't available for Python 3 (python-excel's porting to Py3 stopped in 2011), which would prevent you from spawning new businesses on Amazon or handling (unfortunately standard) MS Office interchange files quickly.

I actually made the decision to run on Python 2 due to unavailability of these libraries for Python 3 about a year ago (I didn't really care about Python before as I primarily used C++/Java/Asm/GLSL/C#/Haskell in the past) - I was deeply shocked by what Py3 did to its own community and still have that weird feeling that Python is slowly killing itself. I didn't want to write my own Amazon and MS Office wrappers in Python 3 anyway and from all platforms available Py2 seemed to have the least amount of troubles (though I had to write some Py3 code for handling advanced TLS properly).

[fat0wl]

yup. I think this is the reason so much eCommerce software is Java (& I'm sure there is a bunch of C# but that's not my thing). Java moves like a glacier but damn is it ever stable, and for those who wish to be "cutting edge" there is everything-under-the-sun on the JVM -- Clojure, Scala, Jython, JRuby, slimmer-by-the-minute Java APIs. And the community is just working their lil buns off trying to modernize this stuff while maintaining interop / backwards compatibility.

I think the flack Java gets is for not modernizing fast enough, but I think in a way that makes you stop and smell the roses (the roses being many years' worth of badass JVM features and massive flexibility/reliability in software development).

But Ruby & Python (I am an ex-Ruby guy) take this cutting edge thing wayyy too far & end up with a community of people who can't trust them anymore cuz they all had their damn feet shot off. Sometimes I see the rates for Ruby guys & wonder if they're high cuz its so productive or just cuz its very niche & the stack is shite. On principal, it's hard for me to imagine going back down that road after experiencing the JVM (plus, I'd like to get away from web one day...).

EDIT: Funny thing I just realized actually -- all the Ruby rates I've seen tend to be exactly on par with a lot of FUBAR niche near-extinct Java technologies. The only people I've seen do better than both these camps are down & dirty Linux/Oracle/Java guys. Maybe my view is skewed but I always feel like the rock-solid langs will live on & eventually "modernize" (though lets be real, modernize is kindof a BS term for slim API & lots of MVC tutorials). These other techs are at the mercy of market trends.

[bitL]

I worked at SUN in the distant past, the backwards compatibility was the mantra #1. Both latest Java and Solaris were expected to run the oldest possible code, and they considered it one of the most important assets.

For many people in eCommerce and banking the arrival of Java was a "manna from heaven", as before they had to deal with the crazy complexity of CORBA and similar technologies. Java simplified this significantly and added at the time modern easy-to-program and fairly safe multi-threading that allowed software like Apache etc. This made Java famous and gave it a "huge karma" amongst developers. Without Java we probably wouldn't have experienced the massive expansion of eCommerce, and now we are lucky to see all the Java's issues - a good problem to have ;-)

Java and JVM as a consequence have their share of problems that arose from initial design decisions such as type erasure (which .NET's CLR avoided) that still deforms all languages based on JVM to this day. Massive boilerplating, J2EE's unreadable XML mess, complex design patterns invented to cope with the limitations of the language are some of the other problems. The compatibility with the old code is still there though.

I've never played with Ruby, I've heard it's still used by many young web companies for their frontends (like SoundCloud) and have heard they had some release that broke most of the compatibility which alienated many of their users. From this and from Python's example I would suggest keeping both the old way and introducing the new ways in parallel so that developers don't have to rewrite/reinvent the wheel when somebody decides to change a part of the language. This served C/C++ pretty well, adding slowly new features and instead of forcing users to use the "current right way", they allowed to live multiple different approaches together. I think that would have served Python well as well - even Py3 feels at places like a hack, similar to Py2.

[tetha]

Java is simply becoming the next C. For a reason, in my opinion, since the languages share quite a few traits both on the language level and the ecosystem level. Both are fast. Both have a rock solid execution system which sometimes feels backwards compatible to the first punch card made of rock (gcc vs JVM). Both of them have huge communities having significant infrastructural services and other huge code bases developed in them (linux kernel, application servers, databases in both teams). The big improvement of java would be a higher programmer safety.

And to be fair, neither language is the prettiest, shiniest and fanciest for it's time, but they get the darned job done.

[fat0wl]

you (& bitL as well) make some fair points but I actually (maybe because I'm newer to Java?) have quite a different experience with it. The XML setup seems minimal now (sure it's all there but most stuff is now handled with reasonable default configs & annotations).

Take Jax-Rs is a good example. Once you understand it you're just like "Ohhhhh this is the same way Rails/whatever framework handles json requests but I can add in all this non-linear logic without forcing it to play nicely within a presumptuous design pattern". Adding many data sources & logic flows is easy.

Bottom line is that I feel like Java can never really be the next C because fundamentally its much safer and becoming way more human readable. It toes the line well -- you can code minimal stuff similar to the scripting languages, or you can drop years of work/research into optimization for high-transaction algorithms knowing that the platform won't abandon you.

(By the way, I am personally hoping for Clojure or node.js to take over the front-end of the stack & then just using Java to do any heavy lifting that is left over)

[burntsushi]

I just looked at that list and I see several prominent projects that (according to that list) don't support Python 3: MySQL-python, gevent, Twisted, eventlet, oauth2, thrift, nltk, mechanize, etc. I've used more than a few of these libraries over the years.

I mean, yeah, the list is getting smaller. That's understood. But that doesn't render the problem of depending on only-Python 2 libraries moot.

I think the fundamental insight of the OP is that Python X (the purgatory corresponding to code compatible with both Python 2 and 3) is an absolute pain to write. There's plenty you can do to make Python X easier (like the `six` library or ignoring Python 3.{0,1,2}), but the programmer is still left with the burden of writing code compatible with both versions.

There's just no denying that the Python 3 transition is a huge fucking pain that is costing people tons of time and money. With that in mind, it is absolutely reasonable that people are resisting the change.

[illumen]

Twisted is mostly ported, and you can install via pip and pypi. Some of the more ancient parts remain to be finished however.

gevent is ported, but not in an official release.

There is a mysql alternative module.

mechanize is ported, but not in official branch

nltk alpha3 for python 3 is available

There are python3 thrift ports, but the maintainers are slow.

[burntsushi]

Yes, I knew most of that. But the fact remains that they aren't complete yet and therefore can't actually be relied on as a dependency.

(N.B. I seem to recall someone telling me that gevent had been ported over a year ago and that all that remained was to just make a release. So count me skeptical until the port is complete.)

And that's just the most popular projects. Imagine what it looks like with the less popular projects? It isn't pretty. This is what the OP was talking about with the "tail" of Python projects.

N.B. I don't personally have this problem with less popular projects. I've always done my best to avoid depending on unmaintained or inactive projects because they tend to rot as time goes on (irrespective of massive changes like Python 2 and 3). But plenty of other people have this problem and it's definitely not an unreasonable problem to have.

[rcruzeiro]

Off the top of my head: Twisted and gevent. It was actually the lack of a Python 3 gevent that lead my to create one crucial piece of software for my startup using NodeJs communicating with the main Python application passing messages via Redis.

[illumen]

Twisted and gevent are available for python 3.

[rspeer]

Look, I'm a Python 3 supporter, but you should never foist the Python 3 fork of gevent on anyone.

I'm pretty sure that, the monkey-patched way most people use it, it's a happy accident when gevent even works correctly on Python 2. This happy accident doesn't happen in an unofficial fork.

[theshrike79]

Available, yes. Working? No.

  File "/usr/lib/python3.2/site-packages/twisted/internet/_sslverify.py", line 1389
    self._cipherString = u':'.join(
                            ^

SyntaxError: invalid syntax

(Twisted 14.0.0, installed with pip)

[chhantyal]

Indeed, 73% of popular packages in PyPi already support Python 3 http://py3readiness.org

And it's getting better everyday.

[Ihmahr]

Numpy, Scipy, NLTK.

These are really HUGE and I can not do my work without them.

[akerl_]

And of those, only NLTK isn't compatible with Python 3.

[kmike84]

To clarify: NLTK develop branch is compatible with Python 3.2+ for more than a year. NLTK 3.0a4 (http://www.nltk.org/nltk3-alpha/) is not released as NLTK 3.0 because of some pending API changes, not because of Python 3 compatibility issues.

[doug1001]

i use NLTK with python 3.4 (and before that, 3.3); NLTK has an alpha release for pyhthon 3; i have not had any problems either installing it or using it.

[rdtsc]

Ok so why would I risk hitting even one incompatible library or have stuff break because those are dev experimental, untested branches. Why? What does Python 3 offer me except risk of breakage?

[phkahler]

wxPython is a big one for me. But it looks more like it's dying altogether rather than not getting ported to 3.

[fithisux]

Have you heard of python3-wx3-phoenix?

It is not dying. It just gets better.

[phkahler]

I've heard of it, but there has never been an official release. I was looking forward to it for a couple years but I've given up.

[postfuturist]

Did you read the article? The long tail is the critical collection of libraries and most of them will never be ported. People write new Python 2 libraries every day, actually--the list is getting longer, not shorter. That said, gevent is an extremely critical library for a lot of production Python 2 code. There doesn't even seem to be a roadmap to port it to Python 3.

[illumen]

gevent is ported to python 3. I guess a proper release would be soon. See here: https://github.com/surfly/gevent/issues/38

If the libraries are used by people, they tend to get ported. Like most of the most popular packages, and 4799 packages listed on pypi.

Any others you depend on? I'm prepared to roll up my sleeves and help.

[YokoZar]

If you genuinely want to help, I suggest porting launchpadlib to Python3: https://bugs.launchpad.net/launchpadlib/+bug/1060734

This is affecting other parts of Ubuntu, as we don't always install python2 anymore after porting all the default apps to python3. The lack of python3 launchpadlib means apport (the automatic bug reporting tool) doesn't work on some installations.

edit: porting launchpadlib requires first porting lazr.restfulclient, which might be genuinely challenging: https://bugs.launchpad.net/lazr.restfulclient/+bug/1000801

[toma7]

Let's assume a modestly sized Python program uses 8 libraries or so. Assuming 80% of libraries are ported, that means 20% aren't. The likelihood of at least one of your 8 libraries not being ported is 1 - (0.8)^8 = 83%. In other words, most people will find that at least one of their cricitcal libraries is not ported. That is certainly the case for me (Bloomberg Python API, for 2.7, and which, btw, only came out 6 months ago). Inevitably the people using Python come from specific domains, and use at least one domain specific library, so while all the common "usual suspect" libraries are ported, there is usually a very high chance that one critical library is not.

Perhaps the way the entire question should be thought of is: are there actually many properly useful Python 3 libraries that are not available in Python 2?? I bet the above equation would look much better when inverted.

[DRAGONERO]

I'd love to have scapy for Python 3. A lot of ports of it have been started but it seems none of them reached a working state.

[dagw]

There's a large number of significant commercial applications that use Python as their scripting and plug-in language. Non of them that I am aware of are showing any interest in Python 3.

[vram22]

Also: Disqus, written partly in Python, mentioned on the High Scalability site:

http://highscalability.com/blog/2014/4/28/how-disqus-went-re...