[zoogeny]

I was watching a recent talk about the new Mojo language [1]. There is a section on SIMD and how they treat scalar values as a special case of vector operations (around the 33 min time).

It does seem that tensors are one of the core abstractions for modern ML systems. I've heard people joke that AI is just matrix multiplication. Python is such a flexible language that creating abstractions around its syntax was super easy. I believe that was part of the reason Python has come to dominate this space (not the only reason obviously).

I too felt the same as you, but as a distant admirer of Zig. I totally understand that operator overloading can be misused. But I have an intuition that at least providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward.

1. https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM

[xh-dude]

Julia has good ideas, interesting trade-offs here as well. ISTM the deep problem is that there’s so much room for optimization under composition of operations or with invariants not captured in types that the typical LinAlg library will never capture. But I agree most languages should be anticipating that users need something helpful here.

[andy99]

  providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward.

This is why I have started using Fortran for writing AI model inference code. It natively handles array manipulation like python and has a `matmul` intrinsic and compiles into fast code. There are some rough edges, but it's great as a low level matrix programming language which was its original point.

[sevagh]

>I've heard people joke that AI is just matrix multiplication

Not sure where the joke is, today's deep learning based AI models are literally matrix multiplications with learned weights.