Hacker News new | ask | show | jobs
by StreakyCobra 3422 days ago
It is one of the reasons that makes me love Arch: developers are pragmatic. They were always in the early birds for taking decision about the future, like switching to Python3 as default interpreter system-wide, embracing the change to systemd, and now this.

The rolling release aspect imposes to stay as close as possible to upstream versions because upstream does not wait on Arch to update their dependencies to newer libraries versions. This, plus the fact that Arch tries to stay as close as possible to vanilla upstream packages (i.e. less changes as possible), force them to be pragmatic about their choices. They prefer to embrace a change directly when it is happening, encouraging at the same time to contribute upstream for helping them make the transition, instead of trying to maintain a legacy compatibility that can only get worse with the time.

For me the picture is simple here: Arch is mostly used/oriented towards personal computers of power users, and most - if not all - of them are on 64 bit for years now. Instead of wasting efforts to maintain 32 bits compatibility, they prefer to drop it. Side effect will be that people will be encouraged to get a modern computer if not already.

5 comments

The switch to the 'python' command running python 3, as a non-arch user, put me off Arch forever. It just broke everything, which had long assumed 'python' would run python 2.

Not installing python 2, and just python 3, with the name python3, would have been fine.

Arch put us in a situation where it was basically impossible to run python 2 with a #! line, as some distros hadn't yet introduced a 'python2' symlink yet.

So to avoid re-opening the debate of Python 2 vs Python 3, let's take this point of view: The Python 3 language may, or may not, have been an improvement, and the transition from 2 to 3 may, or may not, have been done correctly. Everyone has its own opinion on this. But at the end this problem concerns the Python community.

From the Arch developers point of view, the Python developers decided to release a new version of their language. They were going to support python 2 for a while, but the direction that the language was taking for the future was Python 3, so one day or another Python 3 would become the standard, so doing the switch was the good decision.

As said by @tbranyen the work of adapting shebangs have been done on the Arch side anyway, so I don't see a problem for people developing their projects in Python. Also Arch users are mostly power users (you need to understand quite a bit about linux to install it) so they are usually able to handle python 2 vs 3 errors quite well, especially as it is known than Arch use Python 3 by default.

Except that python says that python 3 should be an executable named `python3` and `python` should refer to `python2` [0]

0: https://www.python.org/dev/peps/pep-0394/

It says right in the pep

"...however, end users should be aware that python refers to python3 on at least Arch Linux (that change is what prompted the creation of this PEP), so python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3..."

There was no guidance before the Arch devs made their choice. There's no reason to change back.

Especially since the goal is that python3 will be default eventually.

"...in preparation for an eventual change in the default version of Python, Python 2 only scripts should either be updated to be source compatible with Python 3 or else to use python2 in the shebang line."

There's a good reason to change back. There is now convention in place, and Arch's approach is obsolete.
It seems to me that Arch established their convention first, so why should Arch "change back" when they where first; why are others not "changing back" to Arch's convention, given that Arch's came first?

Well, the answer is that this is a silly discussion. Arch is doing what Arch is doing because Arch is intentionally forward oriented, sometimes to the detriment of backwards compatibility with the old school (and yes, a distro not having a symlink for python2 is at this point nearly 5 years into being "old school").

Fair point. But Archlinux made the switch in 2010 [1], and this PEP was created in 2011.

[1] https://www.archlinux.org/news/python-is-now-python-3/

Using the official, graphical installer for Python 3 on Windows with no previous version installed creates an executable 'python' that refers to Python 3.

(Normally not a Windows user so I have no idea why this is the case or if by design.)

Windows doesn't support shebang lines, so the exact name/location of the interpreter doesn't matter as much as it does on 'nix.
Recent versions of python ship with a launcher shim for windows that does support the shebang line (to simplify using multiple interpreters):

https://www.python.org/dev/peps/pep-0397/

As you say though, it doesn't rely on the names of the exes.

I work on two programs which either not distributed by Arch, or our users choose to download directly (I don't know).

They both contained Python scripts, and I had to waste my time figuring out why they had broken when I had e-mails from Arch users complaining they didn't work -- and then keep replying to future Arch users with the fix.

I don't know how many hours and days of other developer's time was wasted by Arch with this change, but it certainly wasted a good day of my time over the last few years.

I'm an Arch user, and it literally broke nothing for me. Packages were updated, life went on.

You seem to be complaining because other distros hadn't done the right thing. I don't see how that's Arch's fault. Are we to hold back for the lowest common denominator? Do we need every distro to join together and agree to a switchover date?

There was official guidance from Python on the switchover, this wasn't some maverick decision. It was just moving forward.

Not everyone wants to be tied to ancient stuff for backwards compatibility - if you do, it's your job to deal with that.

My Python 2 scripts on Github start with "#!/usr/bin/python", as do many others. All of these broke for you as an Arch user.

Because Arch did what they did, Python now recommends that Python 2 scripts start "#!/usr/bin/python2" (or the env equivalent).

> Are we to hold back for the lowest common denominator? Do we need every distro to join together and agree to a switchover date?

It's not just other distros; it's the rest of the world, including all of the scripts that people run but distros don't necessarily ship.

The right way to migrate would be to ship and use both /usr/bin/python2 and /usr/bin/python3. Leave /usr/bin/python as a symlink to python2 for a while. Eventually, drop the symlink, but do not replace it with python3. Allow users to opt-in to a compat symlink if they wish. Let stuff catch up. When the expectation that "python" is python2 has faded, then introduce a symlink from python to python3 by default, letting users opt-in earlier if they wish.

> There was official guidance from Python on the switchover, this wasn't some maverick decision.

No, there wasn't, and yes, it was some maverick decision. It was done without consultation with upstream.

(I am a non-Arch distribution developer)

This just feels like culture clash to me - the whole change to Python 3 was about accepting the cost of breaking changes to avoid dragging around useless cruft and bad decisions forever.

The idea that we should then never use `python` to mean `python3` is just so backwards. Sure, in your corporate environment where backwards compatability is suepr important, do that.

In Arch, people have a distro that moves fast and breaks things, and we are fine with that. I'll accept the cost of occasionally having to change a shebang line (although as I have almost never had to go outside the AUR - someone else in the Arch community has almost always dealt with this for me).

> No, there wasn't, and yes, it was some maverick decision. It was done without consultation with upstream.

Fair enough, I got my timelines mixed up - but to be fair, it was then ratified as the correct way to do things because the upstream project agreed with the core idea.

> The right way to migrate would be to ship and use both /usr/bin/python2 and /usr/bin/python3. Leave /usr/bin/python as a symlink to python2 for a while. Eventually, drop the symlink, but do not replace it with python3. Allow users to opt-in to a compat symlink if they wish. Let stuff catch up. When the expectation that "python" is python2 has faded, then introduce a symlink from python to python3 by default, letting users opt-in earlier if they wish.

I don't really see why I should have to do this manually? If you don't want that behaviour, use another distro. This just feels like you feel like you should have some say in how other people set up their systems.

If you really hate it so much, don't support Arch - that'd be fair. Trying to shame the distro for doing it ignores the fact it's the core idea of the distro to do stuff like that, that's the point.

> The idea that we should then never use `python` to mean `python3` is just so backwards.

I did not present any such idea. I presented a sane migration path to where `python` means `python3`.

> ...it was then ratified as the correct way to do things because the upstream project agreed with the core idea.

No. It was because the upstream project had their hand forced and are pragmatic about these things.

> I don't really see why I should have to do this manually?

In my proposal for a sane migration path to the ideal? You wouldn't have to do it manually. The distribution would do it. As a user you'd be able to override it to speed up the migration for yourself if you chose; that's all.

I chose to run a bleeding edge distro to be on the bleeding edge. Requiring me to make the changes to be on the bleeding edge manually makes no sense.

Expecting that is as weird as expecting super stable LTS releases to run the latest-and-greatest of everything.

> My Python 2 scripts on Github start with "#!/usr/bin/python", as do many others. All of these broke for you as an Arch user.

So you made certain assumptions. Have you made your scripts use python2, which is safer anyway, everything would've been fine, don't blame Arch for your incorrect assumptions.

No. Before Arch made their change, that /usr/bin/python is Python 2 was the only possible assumption that could be made. No distribution shipped /usr/bin/python2 at that time. There was no need, because /usr/bin/python was always Python 2.

It was only because of what Arch unilaterally did that forced the community to start providing a /usr/bin/python2. Before that, /usr/bin/python2 did not commonly exist at all.

> It was only because of what Arch unilaterally did

Arch is a bleeding-edge, latest software kind of distribution. They did not made the decision 'unilaterally'. They made the decision to ship the latest software for their own distribution, as they always do. It may have proven that a lot of software is extremely poor/inflexible, (i.e. rests on weak assumptions), but I am glad Arch moved forward as that showed their resolve to move technology forward even in the face of a lot of pressure from outside groups.

I just think that instead of blaming Arch, try to make your scripts more robust, i.e. loop on all 'python*' binaries in /usr/bin and use the first one whose --version gives you '2.x' instead of relying on /usr/bin/python being python 2 when the latest version is 3 and Arch is known to ship latest software.

The decision is pretty simple: if a user doesn't like the way Arch does things, they're free to use any of a hundred other distros. I'm with you, I think this is fine.
> Are we to hold back for the lowest common denominator?

Well, the fact that over a decade after the release of python 3 we're _still_ arguing about it shows that lot of people think we should.

I thought the official advice was that 'python' should remain 'python2'?

from https://www.python.org/dev/peps/pep-0394/ (not yet replaced/update) "for the time being, all distributions should ensure that python refers to the same target as python2 ."

Archlinux made the switch in 2010 [1], and this PEP was created in 2011. [1] https://www.archlinux.org/news/python-is-now-python-3/
What I wonder is why one would assume the python command would run python 2, instead of running python 1, assuming python 1 was a thing. By your logic, shouldn't python2, run python3 run python 3, etc. What I'm getting at, is we get used to things being a certain way, but often we're just following a convention that was set in place before we got onboard. Why do we still type "bash" instead of "bash4"?
The reason you would assume it is that, for a long time on every Linux distribution, there was a single way to run python2, and it was called 'python'. I'm not sure what you are getting at, this was just the way it was. You couldn't run python 2 any other way in some linux distros.

The reason we don't need a 'bash4' is that, by and large, bash tried very hard to make sure they didn't break any bash3 scripts, so there is no need for old bash3 scripts to know they are now running in bash4, unlike python3.

I get the point, python 2 scripts aren't fully compatible with python 3. I've written a little python and know that to be the case.

However, I can predict with near certaintly, if python 4 doesn't break python 3 scripts, people will complain if "python3" doesn't invoke python 4, because they've always used "python3" to run the new python and "python" to run python 2.

> Why do we still type "bash" instead of "bash4"

Because Bash was made by, and is maintained by, people who understand how backwards compatability in their program is important.

Just like python. That's why they bumped the major when release backwards-incompatible changes, which is the right thing to do.
Why not be explicit with 'python2' and be done with it?
I agree, but I'd guess the chief complaint is that doing so requires the `python2` symlink to be available. There's at least one claim up thread that it doesn't exist on all systems (create it yourself?). I didn't catch the distribution used, but I DID check an older Ubuntu image I have (I run Arch) and there's definitely a python2 link pointing to python2.7 (`python` also points to python2.7 and there's a `python3` symlink that does what you'd expect). So outside some LTS build, it seems surprising to me that it wouldn't exist in 2017 on a modern distro. I still can't shake the feeling that the thread started around a relatively insignificant complaint (dumping Arch because they use Python3 by default? Come on...).

Personally, I'd rather just use `/usr/bin/env python2` to invoke the appropriate interpreter (if you don't need to pass arguments to it), and if the symlink doesn't exist, the onus is on the user to create it. This is ESPECIALLY true for older distros.

Plus, for a sufficiently complicated application, why would someone want to pollute the global site-packages anyway? I'd rather manage most things inside a virtualenv or similar (pick your Python version), because I've seen far too many install_requires/requirements locking to a specific library version.

In 2017, indeed everyone has added python2, but I assure you it wasn't that way when Arch originally did the python3 -> python rename.

While this is a small issue, this original thread used it as an example of how Arch is good and pragmatic. To me it seemed like the opposite -- it broke lots of code and packages for no good reason. Why should users of older distros have to add a symlink, when before Arch everyone could be sure that '/usr/bin/python' if present would be python2, and '/usr/bin/python3', if present, would be python3?

Now I agree that using virtualenv or similar is a good idea, but this actually caused the most problems for little 20 line python scripts, because often the users of those didn't even realise they were using python.

> I assure you it wasn't that way when Arch originally did the python3 -> python rename.

I disagree. I don't believe it was as painful as some make it out to be. The official packages were updated within a relatively short period of time, and while the AUR packages took longer they're also not officially supported. There was also a news announcement [1]. Honestly, I think of a few other transitions over my years of using Arch as being far, far more painful. The Python2 -> Python3 defaults change isn't one of them.

Although I would agree that Gentoo's approach was somewhat better via eselect, which I believe predated Arch's migration.

> Why should users of older distros have to add a symlink

You're right. That should be the responsibility of the package maintainer. Conversely, why should I, as a developer, have to continue assuming `/usr/bin/python` points to any particular version of Python when most distributions have migrated away from this anachronism? sed does magic, and you really should be using the appropriate symlink for your desired version anyway. Python 3 isn't new. I have a hard time seeing this as problematic because 1) things change and 2) the solution is easy. I'd be happy to entertain a use case where this actually has presented material difficulties, however.

> but this actually caused the most problems for little 20 line python scripts, because often the users of those didn't even realise they were using python.

Well, I do agree: It's a problem for the end user, but the solutions aren't difficult (if someone's using Linux, they probably know enough to fix it). So, either change the hashbang, add a symlink, and--failing that--complain to the package maintainer if it's been packaged up by the distro and the upstream software itself has changed (it should).

My recommendation for developers would be to use the appropriate /usr/bin/python* symlink. Nearly all distros have them in place now (including Ubuntu from a couple years ago). Perhaps assuming /usr/bin/python will always point to a fixed version is a matter of misplaced assumptions.

[1] https://www.archlinux.org/news/python-is-now-python-3/

> for no good reason.

There was good reason, to move to the latest version of a particular software, since that's kind of the whole philosophy of Arch, in fact the reason I originally switched to it in 2011 was because I had enough of me not being able to use the latest software on Ubuntu.

> Why should users of older distros have to add a symlink, when before Arch everyone could be sure that '/usr/bin/python' if present would be python2

If you made no effort to detect the actual version, it's wrong to assume that. It could've just as well been python 1.

You're right. - If Arch is targeted for power users, what's preventing them from configuring symlinks etc? - Why aren't they using virtualenvs? I never install Python packages outside of a virtualenv.
> I never install Python packages outside of a virtualenv.

Global site-packages is all fun and games until someone loses an eye (or a dependency).

I had a fun learning experience with a large Python package once while trying to mix and match dependency versions, simultaneously trying to avoid clobbering the ones already installed by pacman. Without a virtualenv, it quickly becomes impossible to guarantee you won't break something in the process of installing something else. I never made that mistake again.

It really wasn't that bad and the AUR + official channels had patched packages with the correct shebang line. The worst offender for a while was the node-waf builder, had to patch it manually to get any packages to build, otherwise you needed a command line arg every time. This was like 5 years ago tho... These days I have never hit an issue w/ a properly installed python package.
Those scripts should probably be using something like the following, if they really are python2-only: #!/usr/bin/env python2

I get your frustration though, it's a change that breaks a whole lot of existing scripts.

That doesn't address the part where python2 didn't exist in a lot of places at the time, so scripts using python2 to work on Arch would have to be adjusted to work elsewhere. There was no universal solution.
The thing is, python guidelines (PEPs) stated that shebangs should specify python2 or python3 explicitly during the transition period. Using "#!/usr/bin/env python" for a project that was python2-only was not following guidelines, so the breakage was really the dev's fault.

Also, the python2 symlink should have already been there. Bad packaging on some distros resulted in bad practices packaging by some devs. Don't blame that on Arch.

You're getting the ordering of events wrong. Arch's actions weren't in response to the PEP. The PEP was written in response to Arch's actions.

> Also, the python2 symlink should have already been there.

Nope. There was no such convention or specification at the time. There was, however, convention and specification (in the form of "what Python upstream build systems do") on what /usr/bin/python means, which is "Python 2".

> There was no such convention or specification at the time.

There was no formal specification, but there was a convention that many distributions were already following (and quite a few others were not).

Well, even Ansible folks started to switch to python 3 (https://docs.ansible.com/ansible/python_3_support.html). Pushing people off python 2 is not "absolutely evil".
I have no problem with Arch having a "only install python3 by default" rule. They can even not distribute python2 for all I care.

But, don't make /usr/bin/python run python3, that just confuses programs which have made the (reasonable based on past experience, and official python advice) assumption that it /usr/bin/python will run python2.

So forever into the future $ python should run python2? That's not pragmatic at all.
On Ubuntu, "sqlite" starts sqlite2.x; if you want sqlite3.x, you must type "sqlite3". The last version of sqlite2 came out in 2005. As far as I know, hardly anyone still uses sqlite2 for anything anymore. (I could be very wrong though.) Nevertheless, people must still type that extra character "3". The sky has not yet fallen :)

It seems to me that the same approach could work for Python.

Why not? Can you elaborate?
Pythons 4, 5, 6..., N will feasibly exist in the future. Why should Python 2 forever retain the executable name 'python'?
Lets hope Python 4 comes out soon so you guys can unite against the real enemy :-)
Ubuntu does this now with Xenial.
Ubuntu uses Python 3 where possible, but /usr/bin/python is 2.7.
In fairness, this was back when we all thought Py3 would take over Py2 "any day now"™
"back then" the decision made even less sense, since a lot of efforts to make Python 3.x code backwards compatible did not happen yet. Like for instance the u'' literals introduced in 3.3.
Why didn't python just opt to have the python 2 and 3 interpreter in the same binary? It could have detected if a file was python3 with a magic comment or something (overridable with a -2/-3 option), or otherwise assume python2. Or even simpler, have a wrapper script just called 'python' that checks if the file you are calling is python2 or 3 and calls the appropriate binary.
"Explicit is better than implicit." is the Python mantra. Although it's not often taken as literally as some would like, I think it's appropriate in this case. After all, magic detection sounds good on the surface, but it's liable to be problematic and bug ridden.

You don't need magical file-specific juju or a wrapper script on most systems these days to select the Python version. Just call python2 or python3 as appropriate; the symlinks usually exist depending on packager, distro, etc. The problem up-thread appears to be with calling the default `python` on $PATH. Sometimes it points to Python 2.x, sometimes it points to Python 3.x. Sometimes it might even change (Gentoo's eselect, IIRC). I don't see the problem being explicit with which version is needed in the script's hashbang. Python 3 isn't exactly new, and I remember when Gentoo and Arch both migrated to versioned symlinks. It wasn't that painful.

Or switch to a less pragmatic distribution.
Indeed.
> Side effect will be that people will be encouraged to get a modern computer if not already.

All those netbooks (atom processors; 1 GB or 2 GB ram; 32 GB hard drive) would be great with Linux (once the drivers get fixed) and really benefit from 32 bit because of the ram limitations.

Now? They'll end up in landfill.

It looks like the community had the same thought. I follow the arch-ports mailing list and there is discussion right now about a community maintained 32-bit port. They've registered a github account (https://github.com/archlinux32) but there isn't any activity yet.
There are other distributions better suited (even specialized) to be run on old hardware.
Which ones provide a similar approach to Arch?

A six-year-old 32-bit processor (e.g. an Atom in an EeePC 901 or HP Mini 311c) is more than capable of running a modern desktop system.

For a "similar" approach, I would opt for a slackware or a gentoo.

I understand that some people would like to keep Arch 32 bit for their use cases. But as any open source project, the developer are mostly working on it on their spare time, you can not force them to work on anything. Having followed a little bit the discussion on the mailing list [1], I don't remember having seen a lot of objection among the developers. Also, they encourage the creation of a community around i686 [2]:

     However, as there is still some interest in keeping i686 alive, we would like to encourage the community to make it happen with our guidance. The arch-ports mailing list and #archlinux-ports IRC channel on Freenode will be used for further coordination.
So if there are people willing to do the job, Arch will still be usable on 1686.

[1] https://lists.archlinux.org/pipermail/arch-dev-public/2016-D... [2] https://www.archlinux.org/news/phasing-out-i686-support/

Debian testing is similar in "rollingness", Slackware and Alpine are similar in the sense of "no frills". OpenBSD is a good non-Linux choice to try out.
FYI It's actually 8 years old for the EeePC 901 and the last 32-bit Atoms came out in 2009.
The processors are now 64 bit, but the low ram means you're going to suffer if you use a 64 bit OS.

Also, for some of these machines they use a 32 bit UEFI, so even if you want to use a 64 bit OS you'll need some weird tinkering to get it installed.

I erred on the side of caution. :)

I've successfully run Arch on a set of 10 EeePC 901s for a club. Linux plus Arduino programming = awesome kids.

There are brand new machines. Some haven't been released yet.

They come with a tiny bit of ram 2 gb, 4 gb if you're lucky.

The processor is 64 bit, but because of the small ram you'll suffer with a 64 bit OS. (Both Microsoft and Arch wiki agree here).

I don't think arch has ever been particularly focussed on writing an OS for the least performant hardware in circulation.
The "correct" solution to this problem is the x32 ABI [1]. I wouldn't be surprised if it's not well supported in the real world, however.

[1]: https://en.wikipedia.org/wiki/X32_ABI

Absolutely! I have a perfectly good little netbook currently running lubuntu 32-bit. If other distros were to follow suit then they would quickly render my netbook useless.
I run Arch on most everything -- up to now it's worked great on old hardware as long as I'm judicious about what I install. When they stop updating 32bit, I suppose I'll switch to Tiny Core. Or rather, well after they stop updating 32bit and Tiny Core catches up.
Debian will likely support 32 bit until the heat death of the universe, and Slackware probably will as well. There are also the BSDs; OpenBSD and NetBSD in particular will enjoy a long life on non-x64 hardware.
That's a good point about the BSDs. I've been meaning to try a BSD for a while -- it sounds like a good fit for my fleet of aging thinkpads.
I've had excellent results running OpenBSD on old laptops. I have a Dell Latitude CPx (Pentium III) that is 100% supported and runs very fast (relatively speaking) under that OS. I ran it on a Latitude D400 and D620 as well until I gave those machines away (I'm a bit of a Dell collector if you can't tell), and it works fairly well on an Acer Aspire One netbook I keep for when we go camping and my wife wants to have a machine to write her book reviews.
How well does suspend/resume work? I tried FreeBSD on my old laptop and couldn't get that to work. Its the only thing keeping me from trying again..
Good point about other distros catching up. Arch will be updating 32-bit until at least November and it could be another year or more until others like Tiny Core or Debian catch up to the package versions that Arch left off with. Will that now 8+ year old hardware still be worth using enough to switch distros when it is 10+ years old?
> (once the drivers get fixed)

I suspect this means they'll end up in the landfill regardless.

Install an earlier (supported) version of Arch, or use a different distro are the options to try before a landfill.
That's not how Arch works... ;)
How does Arch work?
Arch is rolling release; there isn't really an older version that you can go back to, the installer just fetches the latest versions of the packages.

Of course, there's the Arch Linux Archive, which lets you sync packages as they were on a particular date.

Obviously, but there are also full offline images. Have you never used Arch before?
> They were always in the early birds for taking decision about the future

RHEL7 already dropped support for 32-bit hardware (although it still maintains the compat toolchain+libs for 32-bit binaries on 64-bit O/S).

I believe Windows Server 2012 similarly only runs on 64-bit hardware.

>They were always in the early birds for taking decision about the future

What. It was Arch who

* stuck to SysVinit and told users "you want Systemd, make a package" until long after the big distros had made the switch to Systemd/upstart (https://en.wikipedia.org/wiki/Upstart)

* refused to sign packages properly, because "our users don't need that"

It's exactly that kind of decision that makes me stay clear of it. They have the latest packages, but on the system side changes take way too long.

Arch made systemd the default in October 2012, and added it to repos in January 2012. There's only one "major" OS that was earlier, which is Fedora - to be expected given that systemd is a red hat project.

Suse was October 2014. Debian and Ubuntu were April 2015. RHEL was June 2014.

openSUSE 12.1 had systemd by default, released in Nov. 2011.

https://en.opensuse.org/Archive:Portal:12.1

Ubuntu had Upstart in 9.04 (2009), as did others before they used systemd. I posted a source...