[aseipp]

In general I kind of agree with this, but this move isn't anything specific to Apple. Every company designing ML accelerators is doing it. None of them expose anything but the most high level framework they can get away with to users.

I honestly don't know of a single company offering custom machine learning accelerators that let you do anything except use Tensorflow/PyTorch to interface with them, not a chance in hell any they actually will give you the underlying ISA specifics. Maybe the closest is, like, the Xilinx Versal devices or GPUs, but I don't quite put them in the same category as something like Habana, Groq, GraphCore, where the architecture is bespoke for exactly this use case, and the high level tools are there to insulate you from architectural changes.

If there are any actual productionized, in-use accelerators with low level details available that weren't RE'd from the source components, I'd be very interested in seeing it. But the trend here is very clear unless I'm missing something.

[my123]

Habana has their own SynapseAI layer that their TF/PyTorch port runs on. Custom ops are supported too, via a compiler targeting the TPCs, using a C language variant.

Oh, and they have an open-source UM software stack for those but it's really not usable. Doesn't allow access to the systolic arrays (MME), only using the TPCs is just _starting_ to enumerate what it doesn't have. (but, it made the Linux kernel maintainers happy so...):

https://github.com/HabanaAI/SynapseAI_Core#limitations (not to be confused with the closed-source SynapseAI)

[aseipp]

Well, that's good to hear at least! I knew there was some back and forth between the kernel maintainers recently due to all these accelerator drivers going in without any usermode support; Habana's case was kind of interesting because they got accepted into accel/ early by Greg, but they wouldn't have passed the merge criteria used later on for most others like Qualcomm.

Frankly I kind of expected the whole result of that kerfuffle to just be that Habana would let the driver get deleted from upstream and go on their merry way shipping drivers to customers, but I'm happy to be proven wrong!