[Wowfunhappy]

Note how despite not coming with the OS, `python3` is still placed in the Apple-only `/usr/bin/` directory. And a binary exists at that path either way, it's just a stub by default. Because none of this is confusing at all.

I don't like how Apple handles this. If you have the developer tools installed, it's not especially obvious what is part of those tools and what is actually bundled with the OS. And so it's not surprising that some scripts I've come across will silently break in semi-odd ways without the dev tools.

Also, why is Python a developer tool, but Perl isn't? Or ZSH, for that matter? Does the distinction actually make any sense?

[kbenson]

> Note how despite not coming with the OS, `python3` is still placed in the Apple-only `/usr/bin/` directory. And a binary exists at that path either way, it's just a stub by default. Because none of this is confusing at all.

There's long been problems with this on UNIX systems. For a long time there's generally been three categories of software to track (OS provided, third party managed, manually compiled/installed), and two different locations to put them (/usr, /usr/local. / is different and has/had other constrains and reasons behind usage).

OS provided packages go into /usr, stuff a user manually compiles and installs goes into /usr/local, but where does a third party package manager put stuff? It's not OS provided, but it is packages and maintained, so /usr seems valid. At the same time, it's user installed, so /usr/local seems valid. Both make the other used of those locations more complicated to reason about.

> Also, why is Python a developer tool, but Perl isn't? Or ZSH, for that matter? Does the distinction actually make any sense?

Possibly because some are used by the OS to actually do things for the OS (not sure if Apple finally got rid most those dependencies or not).

The problem of the OS shipping a component for it's own needs that gets used by others is a decades old problem at this point. It's complicated because if it's available users to use, they'll use it, and when the OS wants to update or change it in some way it can break user assumptions. If users update it system wide, they can break OS assumptions.

[throw0101a]

> but where does a third party package manager put stuff?

/opt

> The use of /opt for add-on software is a well-established practice in the UNIX community. The System V Application Binary Interface [AT&T 1990], based on the System V Interface Definition (Third Edition) and the Intel Binary Compatibility Standard v. 2 (iBCS2) provides for an /opt structure very similar to the one defined here.

* https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/opt.htm...

MacPorts uses /opt/local for example.

[kbenson]

Yes, that's where I would suggest putting it, but the reason projects don't is either because they aren't aware of it, feel it's a bit more convoluted to make sure you update where PATH is set, or that /opt is generally used for singular applications, not an extended hierarchy such as /opt/local (which is actually a truly weird name, /opt/macports/ with /opt/macports/bin, /opt/macports/lib, etc would make more sense). It makes sense to use /opt from an "application" point of view, but I think the problem is many of these package managers see themselves as more than that, and as pseudo-system components, which clouds their reasoning.

The LFS spec you linked to is actually ancient at version 0.65, the latest is 3.0 and available here[1]. It's a bit "do what you want", but it really expects the "local" part of /opt/local that MacPorts uses to be some sort of identifier for MacPorts.

That said, it's the Linux Filesystem Hierarchy, so it only applies to Apple stuff inasmuch as it's common practices for UNIXes in general. And it's not like the major Linux distros follow it fully either. They're happy to break from it where it preserves some historical layout their OS has had that they think is easier for their users/admins.

1: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html

[SulphurCrested]

python3 is a developer tool because it is the extension language for the lldb debugger. For example, I have python3 scripts for formatting my custom C++ data types, and that formatting also works in Xcode's visual debugger (which is presumably using the lldb API).

Previously, lldb's extension language was python2. This changed a release or two of Xcode ago. At that time, to use lldb scripts with that new release of Xcode you either had to run on Catalina (which added python3 to the existing python2) or install your own python3 and point lldb at that. This actually was what pushed me into moving to Catalina on my development system, because the Unicode handling would be much easier in python3.

As I understand it, Monterey and the Xcode that is released for it move the necessary python3 installation from the operating system itself to Xcode. I'm fine with that.

(I would not be fine with Apple telling me I have to go waste my time figuring out how to install a suitable python myself, when I never had to before. And indeed, many of their own developers may be like me and know a little Python but are in trepidation of the env thing.)

The way it was before meant that if they wanted to update python they had to release a new version of the operating system. This is clearly an imposition on the vast bulk of macOS users, who are not developers.

[samatman]

The reason for zsh is simple enough: Terminal is a standard Mac app, and there are some system maintenance things an ordinary user might do which require the command line.

So there needs to be a shell, and that's zsh now.

Apple switched to zsh because they're allergic to later GNU licenses and the version of bash they were shipping was absurdly out of date.

[Wowfunhappy]

Yes, that's the practical reason. But what's missing is a conceptual reason, something that makes it easy to remember what is and is not a developer tool.

Possible approaches that would be less confusing than what we have today:

1. Ship Python 3 with macOS.

2. Ship all of the command-line developer tools with macOS, including clang et al.

3. Bring back the Snow-Leopard-era /Developer folder, and put developer tools there.

4. Put developer tools in /usr/developer, or /usr/local/developer, or /usr/local/Apple, to balance both UNIXness and clarity.

5. Don't ship Python at all. The Python foundation already distributes a .pkg installer for Mac. If xCode needs Python, follow your own Human Interface Guidelines and put it in the Application bundle.

All of these approaches have downsides, but I still think they'd be clear improvements over the current situation. Which is to say, I want Apple to figure this out!

[szc]

It isn't an allergy.

At the time the GPLv3 was written, the GPLv3 was deliberately crafted to put commercial companies into a conflict situation.

(A) Give up selling audio with DRM (DRM did go away) (A) Give up selling video with DRM (A) Give up using HDMI as an output port (A) Give up using digital signatures to secure the integrity of software

-OR-

(B) Give up using GPL v3 software, GCC, Bash

Pretty sure most customers actually want many of the things in (A) and a minority actually care about (B). The items in (B) can be post-installed by those that really want it. Sure those in (B) have a loud voice, but, HN, are they really representative (democratically a majority)?

How would you make the choice between (A) or (B)?

I would bet that if a subeoena for FSF/GNU email was issued there would be a lot of messages related to manipulation and coercion for licensing and re-licensing. The sort of stuff, when associated with companies turns into scandal, lawsuits and monopoly investigations. (GCC almost got to be what could be considered a monopoly compiler)

I do believe that open software has a place and is a really good choice. My opinion is that the GPLv3 is completely toxic, spawned from negativity and is ultimately anti-open source. I will never release any software under the GPLv3.

[Wowfunhappy]

> Give up selling audio with DRM (DRM did go away) (A) Give up selling video with DRM (A) Give up using HDMI as an output port (A) Give up using digital signatures to secure the integrity of software.

Does GPLv3 really prevent an OS from doing those things? I think it would just prevent it in the same executable as GPLv3 software?

I don’t actually understand Apple’s “allergy”. On the iPhone I’m sure they object to the Tivoization clause (which coincidentally is why I like to use GPLv3), but I’m pretty darn sure macOS doesn’t fall afoul of anything...

[zitterbewegung]

Python 3 is required for the ability to install coremltools which makes it a developer tool.

Python 2.7 might be there because old software is there for compatibility reasons for software possibly for mathematical software that never really gets updated. Removing it in future versions makes sense because of the fact that if you haven't updated it won't work on ARM Macs.

The best way to make someone upgrade is to deprecate and then remove access. At minimum the worst case scenario is they roll their own and install Python 2.7.

[szc]

#!/usr/bin/python

How exactly is the above supposed to work when /usr/bin/python doesn't exist. If Apple chose to break the #! contract by doing something else, how would you feel about that?

[Wowfunhappy]

The scripts should be using #!/usr/bin/env python

[szc]

This answer is both a great answer, but also a terrible one.

This answer implies all "python" binaries across all operating systems and distributions for all time, are backwards and forwards compatible, no work needed. Guaranteed 100% equivalent.

What about Python2 v.s. Python3?

This isn't true and cannot ever be. This happens for other scripting and interpreted languages.

This approach means that the burden of choice and setup is transferred 100% to the person running the script. If you have 2 scripts that require different dependencies, then you will encounter this problem. I think it is this that encourages folks to include hard coded paths to enumerate explicit dependencies.

Lots of the "package" managers for these scripting languages also don't deal with this very well. They advocate a "do it my way" or "do it yourself". Different languages do it different ways.

Ultimately, the person wanting to run the script, just wants to make it run -- they will follow the instructions to make it work and along the way will make "global" changes -- which will impact what will happen for any other script in the future. This will likely be diverged from anyone else who has a "base install".

The above is mostly about my observations with using Ruby, not Python. However, few attempts in using pre-packaged complex python "recipes" has always resulted in similar conflicts.

[Wowfunhappy]

> This answer implies all "python" binaries across all operating systems and distributions for all time, are backwards and forwards compatible, no work needed. Guaranteed 100% equivalent.

Practically speaking, how does using env versus the absolute path make this any better or worse? You as the script author don't know what version of Python/Ruby/Bash/etc lives in /usr/bin. Maybe you could do some kind of automagic detection based on the user's OS, but any such assumptions are likely to go stale over time.

All env does is give users a choice about where to put their binaries. The versioning situation is a real problem, but I don't think using env makes it better or worse.

[comex]

Nit: It’s not that /usr/bin/python3 gets replaced. It’s always a stub, but the stub execs the real python3 if found.