[smspf]

A couple of years ago we reached out to the python community about wheels and arm64 - how it should be handled and whether they plan on embedding non-x86 blobs. We received the standard "we'll think about it and let you know". Now that Apple switched to arm64, all communities are suddenly interested in porting things to arm64.

And of course, Apple is not investing in these ports, at least as far as I know. They just rely on what other arm64 players did in the ecosystem before Apples rolled out M1; respectively lets developers figure out the remaining porting.

As much as I hate to say this, IBM does handle porting things to ppc64 right - you can find IBM contributed code and optimizations anywhere you look. For many packages, porting to arm64 was a matter of "does it have ppc64 support? if so, it can be reused for arm64" ...

Disclaimer - used to be a contractor porting stuff to arm64 for a couple of years.

[oefrha]

> And of course, Apple is not investing in these ports

What count as "investing in these ports"? Does submitting patches to CPython (and many other open source projects, including NumPy) for macOS 11 and Apple Silicon count? Here's a list of Apple-submitted PRs on python/cpython: https://github.com/python/cpython/pulls?q=is%3Apr+author%3Al... Also some co-authored patches excluded by the search. See https://bugs.python.org/issue41100 for more related PRs. Your subsequent comments about IBM seem to imply that these do count.

I'm also curious about who's "we" in "we reached out to the python community about wheels and arm64".

---

Edit: Forgot to say, arm wheels have been supported for many years now (not sure about the specific timeline of aarch64 support, but if 32-bit arm was supported I don't see why aarch64 wouldn't be). Maybe most famously there's https://www.piwheels.org/ for RPis. Are you talking about aarch64 support on PyPI/warehouse?

[smspf]

By investing in these ports I was thinking about any contribution that enables/improves arm64 support. So yes, PRs for python itself definitely qualify. However, the python ecosystem is not only python, there are lots of python packages that still lack arm64 optimizations or native arm64 support. Most of them work, being python, but every once in a while you'd run into a package that does not just work on arm64 without some tweaking.

In my case, "we" referred to the working group of ARM contractors working a very specific higher level project (at the time we were doing some cloud/openstack/k8s stuff, which indirectly required some python packages). Note that the way contractors were assigned to projects at the time implied that sometimes you'd run into overlaps, e.g. some packages we needed had already been ported by a different team of contractors for a different project a while back. Sometimes multiple teams would hit the same missing port at the same time, most of the time we could sync and team up by escalating that through the right channels, but sometimes we'd just end up duplicating work items and end up with two different sets of patches/ports.

About wheels, someone else in my team handled that, so I'm not that familiar with the issue - it might have been a particular package that was lacking arm64 wheels or it might have been impractical to use a different repository only for arm64 (in general we'd try avoiding using separate resources based on architecture and we definitely did prefer the arch-agnostic approaches, e.g. debian repository URLs are arch-independent, dockerhub manifests allowed using the same container image names etc.). We were interested PyPI at the time (couple of years ago), maybe the situation has improved meanwhile, but we just accepted compilation at install time as a good enough solution.

[Hamuko]

For what it's worth, I think Apple sent the Homebrew devs ARM Mac Minis for building and when the Homebrew devs asked for more, they sent more. So they're doing something, even if they're not doing as much as one would hope.

[smspf]

To be honest, I don't blame Apple for this. I blame the ARM ecosystem which is very fragmented, each company working with ARM is contributing to the stuff they are interested in and that's it.

Lots of contractors and always shuffling/changing projects they work on.

[oblio]

Isn't Apple doing the exact same thing with their proprietary ARM ISA extensions?

[my123]

Their ISA extension is an ML-specific one, and macOS runs fine with it disabled. Their public compilers do not support it either.

You are supposed to use it through Accelerate.framework, which exposes a more traditional interface to that capability.

[skissane]

> Their ISA extension is an ML-specific one

Apple Silicon has more than one ISA extension.

There is also the x86 memory ordering extension used by Rosetta 2.

Maybe there are yet others too.

[my123]

> There is also the x86 memory ordering extension used by Rosetta 2.

That one isn't really a requirement either, and is handled fully in kernel mode.

Wkdm and friends are handled fully in kernel mode.

APRR? Not a strict requirement for user-mode, JIT regions are just left as RWX without it.

[saagarjha]

There are more but they are not exposed to applications. Apple wants you to ship standard arm64 code.

[simonh]

They also submitted some patches to MacPorts, and I think Homebrew though I couldn't find a reference offhand. So i suppose it's always possible to argue they could do more, but code and hardware is a long way from nothing.

[fouric]

Why would Apple invest in third-party toolchains? Aren't they incentivized to only support Xcode, Swift, and other tools that are as tightly bound to their ecosystem as possible?

[bla3]

Because they sell hardware and commodize its complement. Having a good Dev experience is part of that, be that a good story for mac-only software (swift/xcode -- ymmv if this is better than other technologies, but that's the intent) or having your posix software mostly just work.

[thetinguy]

Apple uses community tools internally for some things.

[m463]

They can't. They've put themselves on the path where they can't abide by the GPL. GPL3 says you must preserve the right to run the software and apple takes that away from their customers then sells it back.

So apple nopes out of it. They rely on the initial boost they got from GPL software. They do some MIT/BSD licensed stuff (and publish some of it but not everything). Every year less and less happens, unless it's their own language/etc.

They do have nice gradients and round corners though. :)

[JimDabell]

> They can't. They've put themselves on the path where they can't abide by the GPL.

PyPy is MIT licensed.

[m463]

I'm talking about apple investing in non-apple/community software.

[oefrha]

You're talking about GPL3 software, which is irrelevant here. They clearly do contribute to MIT-licensed "non-apple/community software", as I demonstrated up thread, so what's the point of this "they can't" BS.

[m463]

just remember it all started with stuff like gcc afaik.

[ghaff]

>They rely on the initial boost they got from GPL software

You mean BSD? That's how Darwin is licensed.

[derefr]

I believe the GP is talking about the software macOS ships with, not macOS itself.

macOS ships with some GPL2 software, along with its BSD/MIT/Apache-licensed Darwin layer. (The GP is presumably implying that having this GPL2ed software "built-in", helped macOS entrench itself as a useful POSIX for developers — at least before tools like MacPorts/Homebrew came along to make acquiring this type of software "after-market" simple.)

For any GPL2 software macOS makes use of that then transitions to GPL3 licensing, Apple can't actually adopt the GPL3ed version (exactly because of the "runs anywhere" clause in the GPL3) and so instead, macOS keeps the old, GPL2ed version of the software around, left to rot. Eventually, if it's something critical, Apple removes their dependency on said software, replacing it with something that's not GPLed.

This is the story of Bash on macOS: it was regularly updated for as long as it was GPL2-licensed; then it got "stuck" on the last GPL2 version when GNU transitioned Bash to GPL3 licensing. The Bash on macOS remained on that old version for the longest time, before finally being swapped out for Zsh in Catalina.

[m463]

also they were technically in violation of the GPL with bash. They made some modifications and did not release them all. (try to compile bash without rootless.h)

[glandium]

Presumably, they are on https://opensource.apple.com/source/bash/bash-118.40.2/ although sources for the latest macOS are usually not released in a timely manner (no Big Sur at the moment, for instance)

[saagarjha]

rootless.h has never been made available to the public as far as I know.

[m463]

In the beginning the packaged a bunch of stuff, both mit/bsd and gpl. At some point they stopped gpl software. The darwin stuff is published but not everything. The code shared has become less over time even as apple has grown enormously.

[my123]

They didn't stop shipping _GPLv2_ software. They just never shipped GPLv3 software.

[saagarjha]

Emacs is gone IIRC

[saagarjha]

Parts of Darwin are APSL.

[sdoering]

Yes. That. I am done with the MacOS for good. My first Mac was a Classic II, back in '92. My last will be a MB Pro 2018 (actually 2020, but already repaired once and broken beyond working condition again). It was my third MB Pro in 3 years and every single one had to be repaired once before it crashed shortly thereafter. The company I work for has to replace around 1% - 2% of their MB Pro fleet per month. 6 - 10 devices are always in repair (additionally 1 - 2%).

I have notified my superior, that I am not able to work properly in January when I am back from a 2 months LOA were I privately switched everything over onto a WIN10 with WSL2 running Ubuntu.

I am not missing a thing currently. OK - I know at one point I will probably be missing Keynote.