[eridius]

Apple needs the shims anyway, even if the directory was writable. Note that the shims existed before SIP was a thing. The reason Apple needs the shims are twofold:

1. Xcode is distributed as a single application bundle. All of the developer tools are somewhere within /Applications/Xcode.app. Xcode used to be installed with a package installer, so it would dump its tools into the system directories, but a number of years ago they switched to the application distribution for a few reasons, the biggest being so it can be distributed via the Mac App Store.

2. If you don't have the developer tools installed, the shims prompt you to install the standalone Command Line Tools bundle (which goes into /Library/Developer IIRC, still as a single self-contained bundle like Xcode). Without the shims, there wouldn't be any way to trigger an installation of the standalone Command Line Tools.

> Who switches the PATH to /usr/bin:/bin or /usr/local/bin:/usr/bin:/bin?

Someone who's writing a script and wants to be insulated from potentially-malicious entries in PATH. Or shells or shell-like programs executed without any PATH provided at all, which can happen if you invoke them with `env -i`.

> Perhaps because of the same reason why on the average server it's suggested that to keep your attack surface as low as possible you only install the minimum set of utilities and services necessary for the server to be operational?

Huh? What does any of this have to do with attack surface? These tools are on the system regardless, moving them into a different directory has no bearing on attack surface.

> And you claim its system critical, yet it isn't

I don't know if you're deliberately misrepresenting what I said, or if you simply haven't bothered to actually read what I'm saying. These binaries are not critical to the operation of the system. In fact, the system doesn't ever even try to invoke them during normal operation. But they're still part of the system, and if you get rid of them there's no way to get them back without reinstalling the OS.

> and if a non-system binary is replaced and that binary is installed in a location away from system binaries you can put it back?

..what? I genuinely can't figure out what you're trying to say with this sentence.

> Not to mention that only under the current situation only Apple can supply security updates to a third party took they don't actually maintain?

Congratulations, you've just described how every third-party component bundled with the system works.

> And they can't be as quick as a system administrator who wants to remove it entirely?

That's completely nonsense. Any "system administrator" that believes these tools should be removed from OS X should be fired for incompetence. These are shims, and their entire point is to provide a predictable location within the default PATH for a bunch of tools that are actually bundled inside Xcode or the standalone Command Line Tools download. If you don't want to have git/clang/etc on your system, don't install Xcode or the standalone Command Line Tools. It's that simple.

> Because right now if you delete that "system" git the shell died t give an error that the executable can't be found but the shim gives an error? That hardly makes sense.

Huh? You can't delete that "system" git. And as I said above, it doesn't make any sense to even try, because it's just a shim. If you really don't want to have git installed on your system, don't install Xcode.

So really this pretty much boils down to "if you don't want developer tools on your system, don't install the developer tools". All that you'll have are a set of shims that offer to install the developer tools if you try and run them (which I hope you can see is a pretty sensible suggestion, and is akin to how linux distros often suggest packages you can install when you try and execute commands that don't exist).

[chris_wot]

I'm afraid I'm just not clear as to why Xcode needs those shims. Those shims are designed to launch a dialog box prompting to install the command line tools. They are completely unnecessary, because as it turns out bash does this better, but for that you need v4.

The fact that there is a dialog box that comes up means that if a script is run then it should fail if it can't find git. Preinstalled shim executables are a bad idea - I would hope that bash not find git rather than automatically ask to install the XCode Command Line Tools!

>> Who switches the PATH to /usr/bin:/bin or /usr/local/bin:/usr/bin:/bin?

> Someone who's writing a script and wants to be insulated from potentially-malicious entries in PATH. Or shells or shell-like programs executed without any PATH provided at all, which can happen if you invoke them with `env -i`.

Then they should point PATH to the standard locations where the software is installed, which would indicate that if there was a further directory then they should append it to the end. Not entirely sure what the issue is with this.

Somewhat ironically, if they are expecting that git would be the one run in the /usr/bin directory they would find that the shim bypassed the PATH variable entirely and executed a binary in an entirely different directory. So much for their concern over insulating from potentially-malicious entries by reducing the search path to /usr/local/bin and /usr/bin...

>> Perhaps because of the same reason why on the average server it's suggested that to keep your attack surface as low as possible you only install the minimum set of utilities and services necessary for the server to be operational?

> Huh? What does any of this have to do with attack surface? These tools are on the system regardless, moving them into a different directory has no bearing on attack surface.

I never said that. You cannot remove the shims, so in essence you haven't removed the software. If you are forced into keep the shims, even though you don't want to have git installed, you still potentially have it installed. The shims themselves have the most opaque way of finding git I can possibly think of... the following is what I found on it:

"Below are what I've deduced its order of checks to be. Remember that this checking is the responsibility of libxcselect. xcode-select simply provides a way of manipulating some configuration on the system that libxcselect will make use of when one of these shim commands is run, or if xcrun is invoked directly by the user with specific options.

If the DEVELOPER_DIR environment variable is set when a command is run, any system default will be overridden.

If a symlink at /var/db/xcode_select_link is present and it points to a valid developer dir within an Xcode or the CLI tools, this will be used. This symlink will be created when running xcode-select -s <path> if <path> is either an alternate Xcode path or that of the CLI tools, and there is already an Xcode.app in /Applications.

If neither of the above are set, and the path /Applications/Xcode.app/Contents/Developer is present, this is used. If no Xcode path as above are set, but CLI tools are installed in /Library/Developer/CommandLineTools, these are selected." [1]

Note that those are from shims preinstalled on OS X.

So basically, the shim runs xcrun, which relies on xcode-select. xcode-select allows you to set the environment variable DEVELOPER_DIR to whereever you want, and will switch to this directory then run the tool from within there. That's a remarkably convoluted process, and frankly I can see how easy it would be to increase attack surface.

The issue I have with the shims is that there is no way of removing them by default.

>> and if a non-system binary is replaced and that binary is installed in a location away from system binaries you can put it back?

>..what? I genuinely can't figure out what you're trying to say with this sentence.

I did explain myself poorly there. All I'm saying is that any non-critical software can be installed into a different directory. Currently it is installed in a system directory where the entire point of it being immutable is to ensure that no binaries can be modified by anyone other than the installer service, yet those shims bypass this by ensuring that if you run the shim, you actually are running an entirely different executable located on a part of the system where malware does have write access. Thus defeating the purpose of SIP!

>> Not to mention that only under the current situation only Apple can supply security updates to a third party took they don't actually maintain?

> Congratulations, you've just described how every third-party component bundled with the system works.

Indeed - and I'm saying it's a bad situation if the software has gaping security holes that don't get patched quickly or effectively by Apple - and you can't remove it easily.

> > And they can't be as quick as a system administrator who wants to remove it entirely?

> That's completely nonsense. Any "system administrator" that believes these tools should be removed from OS X should be fired for incompetence. These are shims, and their entire point is to provide a predictable location within the default PATH for a bunch of tools that are actually bundled inside Xcode or the standalone Command Line Tools download. If you don't want to have git/clang/etc on your system, don't install Xcode or the standalone Command Line Tools. It's that simple.

What? If you want to install OS X command line tools, then use:

xcode-select --install

What doesn't make any sense is to have a "shim" that invokes a dialog box when you invoke the command to install the XCode command line tools. What you currently have is a situation where if someone has an environment where they might be invoking git in a script then it runs a program that invokes an interactive dialog box prompting to install the XCode command line tools! Aside from the sheer silliness of a dialog box in a shell script, there are a lot of shell scripts that expect that if the system can't find the binary, then the shell will invoke an error which they catch and handle gracefully.

> So really this pretty much boils down to "if you don't want developer tools on your system, don't install the developer tools". All that you'll have are a set of shims that offer to install the developer tools if you try and run them (which I hope you can see is a pretty sensible suggestion, and is akin to how linux distros often suggest packages you can install when you try and execute commands that don't exist).

No, what it boils down to is that if you have a tool that you don't want installed on the system you get a shim binary that executes regardless and makes it harder to handle in your average shell script.

What you don't realise when you speak about how Linux distros suggest packages is that they don't actually run a new executable, they are actually using a version of bash equal to or higher than version 4, which looks for the shell function command_not_found_handle. This gives a number of suggestions and returns status code 127, which is the same code that you get if the shell can't find the executable or that function. [2] There is absolutely no executing of a specific binary, it's all completely done within the user's bash environment.

Apple currently use bash 3.2.57(1) on my OS X El Capitan system, so they don't have the ability to leverage this feature. Which is a shame as it is far and away more sensible that preemptively installing a shim in the directory where the binary would normally be. Which, as far as I can tell, was an inelegant hack done by the XCode install team all the way back in OS X Maverick and preloaded on OS X. It's not scalable in any way, as it relies on the Operating System to know about all possible software that might or might not be installed by the user - having shims for each and every possible 3rd-party program in existence is hardly a good strategy - and not one that Linux distributions follow.

1. https://macops.ca/developer-binaries-on-os-x-xcode-select-an...

2. https://www.gnu.org/software/bash/manual/bashref.html#Comman...

[eridius]

What you've written is an extremely long-winded and incredibly misguided and wrong post, I don't even know how to address all of the problems here, so here's a quick summary:

• Xcode is a self-contained app bundle, it does not install components across the system, which would be required for your scheme. This is intentional for several reasons.

• Having 2 copies of Xcode installed at once is explicitly supported. This requires that Xcode not be installing components across the system or it they would interfere with each other.

• Your description of xcode-select is overly-convoluted. In practice it's really very simple. xcode-select sets the default Xcode installation used for xcrun and the shims. You can override this selection on a per-invocation basis by setting the DEVELOPER_DIR env variable instead. If you don't have the env variable set, xcode-select is roughly equivalent to replacing all of the shims with symlinks that point into the selected directory. And you haven't explained why any of this is supposed to increase the "attack surface".

• You keep claiming that this defeats the purpose of SIP, which makes no sense at all. I can't even figure your logic for claiming it defeats SIP, which makes me wonder if maybe you're just trying to spread FUD.

• I absolutely do know how Linux distros suggest packages, please don't try to tell me that I don't.

• Using bash v4 is not more sensible than installing a shim, because, as I've explained at great length, is required for reasons other than just suggesting the installation of Command Line Tools. You keep ignoring all of the reasons why this is required for current functionality and are focusing on the single aspect, of suggesting the installation of Command Line Tools, that could conceivably be done a different way.

• Besides, saying "use bash v4" is not a solution anyway, because many people do in fact switch their default shell, at which point they'd lose the instructions on how to install the command-line tools.

I'd really rather you don't give me another long-winded reply that boils down to you saying "I don't understand the point of SIP or how it works, or how Xcode's self-contained app installation works".

[chris_wot]

> • Xcode is a self-contained app bundle, it does not install components across the system, which would be required for your scheme. This is intentional for several reasons.

A self contained app bundle that does not install components across the system and requires access to common executables that redirect access to utilities shared amongst the app bundles.

Doesn't that tend to break the self-contained nature of the packages?

If I was to setup two completely seperate Xcode environments that I needed to run at the one time I think I'd probably investigate a chroot jail.

> • Having 2 copies of Xcode installed at once is explicitly supported. This requires that Xcode not be installing components across the system or it they would interfere with each other.

I'm not following. How does having shims for git prevent this?

• Your description of xcode-select is overly-convoluted. In practice it's really very simple. xcode-select sets the default Xcode installation used for xcrun and the shims. You can override this selection on a per-invocation basis by setting the DEVELOPER_DIR env variable instead. If you don't have the env variable set, xcode-select is roughly equivalent to replacing all of the shims with symlinks that point into the selected directory. And you haven't explained why any of this is supposed to increase the "attack surface".

I thought this was rather obvious... Apple don't want people to symlink to other applications from /usr/bin just by exporting an environment variable.

  630188s-iMac:~ 5K$ mkdir -p ~/xcsl/usr/bin
  630188s-iMac:~ 5K$ echo '#!/bin/sh' > ~/xcsl/usr/bin/xcrun
  630188s-iMac:~ 5K$ echo 'echo "Hello world!"' >> ~/xcsl/usr/bin/xcrun
  630188s-iMac:~ 5K$ chmod a+x ~/xcls/usr/bin/xcrun
  630188s-iMac:~ 5K$ export DEVELOPER_DIR=~/xcsl/
  630188s-iMac:~ 5K$ git
  Hello world!

Notice something? No sudo needed.

> • You keep claiming that this defeats the purpose of SIP, which makes no sense at all. I can't even figure your logic for claiming it defeats SIP, which makes me wonder if maybe you're just trying to spread FUD.

See above.

>• I absolutely do know how Linux distros suggest packages, please don't try to tell me that I don't.

You wrote:

"All that you'll have are a set of shims that offer to install the developer tools if you try and run them (which I hope you can see is a pretty sensible suggestion, and is akin to how linux distros often suggest packages you can install when you try and execute commands that don't exist)."

You were suggesting that Linux distributions installed shims in place of where the executables are normally ("All that you'll have are a set of shims [which are] akin to how linux distros often suggest packages you can install when you try and execute commands that don't exist"). All I was saying is that Linux distros don't install shims.

If you meant something other than that I'm not sure what your point about Linux was given they have a very different mechanism for suggesting packages. Probably the biggest difference, even beyond the shim point, is that they don't actually prompt you to install the package...

> • Using bash v4 is not more sensible than installing a shim, because, as I've explained at great length, is required for reasons other than just suggesting the installation of Command Line Tools. You keep ignoring all of the reasons why this is required for current functionality and are focusing on the single aspect, of suggesting the installation of Command Line Tools, that could conceivably be done a different way.

Indeed, I am :-) You haven't really given a good reason why shims are required. I've shown how they can be abused above. I'm happy to agree to disagree though!

> • Besides, saying "use bash v4" is not a solution anyway, because many people do in fact switch their default shell, at which point they'd lose the instructions on how to install the command-line tools.

You mean the people who know how to install and use a different shell to bash? The same people who have worked out how to install packages from the command line, such as xcode-select --install? Or perhaps you mean those folks who use an alternative package installer like homebrew who know that you just have to run brew install git because they ran brew install zsh to install their alternative shell?

I don't think it's going to fuss them too much personally.

> I'd really rather you don't give me another long-winded reply that boils down to you saying "I don't understand the point of SIP or how it works, or how Xcode's self-contained app installation works".

Given I didn't do that, I can't exactly give you "another". I do understand the point of SIP, and I think it's kind of interesting that you think that a shim is required for anything other than prompting the user how to install things like git. You could actually run Xcode without the Xcode command line tools package, and you could quite easily install git yourself and have everything else work.

Anyway, I'm detecting that you are beginning to get a bit personal so I think I won't say anything further other than I respectfully hope we can agree to disagree!

[chris_wot]

Here's a better version:

  mkdir -p ~/xcls/usr/bin
  cat <<EOF > ~/xcls/usr/bin/xcrun
  #!/bin/sh
  if [ "\$1" = "git" ];
  then
     echo "Hello world!"
     exit \$?
  fi
    
  shift 1
  xcrun "\$@"
  exit \$?
  EOF
  chmod a+x ~/xcls/usr/bin/xcrun
  chflags hidden ~/xcls
  echo "export DEVELOPER_DIR=~/xcls/" >> ~/.bashrc
  export DEVELOPER_DIR=~/xcls/

Results in:

  630188s-iMac:testrun 5K$ git HelloWorld
  Hello world!

[eridius]

> If I was to setup two completely seperate Xcode environments that I needed to run at the one time I think I'd probably investigate a chroot jail.

And that's why you don't develop software for actual real human beings to use.

> I'm not following. How does having shims for git prevent this?

Because Xcode doesn't install any system components at all, the system itself provides the `xcode-select` utility that you use to select which Xcode version to use the tools from when invoking /usr/bin/git etc. If Xcode installed crap on the system, that would be impossible because each version of Xcode would have overwritten the other version's utilities. Not only that, but it would also break the ability to have 2 versions of Xcode, period, because the version that had its utilities overwritten would most likely break when the versions of the utilities change out from under it.

> I thought this was rather obvious... Apple don't want people to symlink to other applications from /usr/bin just by exporting an environment variable.

Well no, that's not the goal of SIP. SIP means nobody can muck with the system components. But the behavior you've described is the intended behavior of /usr/bin/git et al. The fact that these utilities are working as designed does not mean SIP is broken.

> You were suggesting that Linux distributions installed shims in place of where the executables are normally

No I didn't, I suggested that the fact that these shims suggest installing Command Line Tools is akin to how Linux distros often suggest installing packages via the package manager. I have no idea how you could possibly imagine I thought Linux added a shim for every single third-party utility in existence to /usr/bin. That's complete nonsense and I'm not sure how to interpret this as anything other than you insulting my intelligence.

> You haven't really given a good reason why shims are required.

I've given you multiple reasons, but you ignore most of them and suggest completely broken workarounds for the others.

If you're going to pretend that I haven't explained numerous times why it works the way it does, I don't think there's any more point in talking to you. You're too focused on your broken solution to a complete non-problem for a system you don't understand in the slightest and you obviously don't want to actually bother to learn why they work the way they do.

> You could actually run Xcode without the Xcode command line tools package

Well yes, of course you can. The fact that you think that's a surprising statement means you don't even begin to understand how Xcode works or any of the stuff I've been explaining. Please stop parading your ignorance around.

[chris_wot]

I wrote an article with my thoughts, and I've posted it to HN:

https://news.ycombinator.com/item?id=11652578

Feel free to comment there further if you want.

Peace :-)