[almostdigital]

Here's the same benchmark with np.matmul instead of native python (on M2 MBP)

    Python             4.216 GFLOPS
    Naive:             6.400 GFLOPS            1.52x faster than Python
    Vectorized:       22.232 GFLOPS            5.27x faster than Python
    Parallelized:     52.591 GFLOPS           12.47x faster than Python
    Tiled:            60.888 GFLOPS           14.44x faster than Python
    Unrolled:         62.514 GFLOPS           14.83x faster than Python
    Accumulated:     506.209 GFLOPS          120.07x faster than Python

[microtonal]

Does that use Apple accelerate? Depending on the matrix size, that sees a bit low, even the M1 Pro can easily reach 2.2 TFLOPS.

[gyrovagueGeist]

What is your BLAS backend?

[andy99]

Yeah this is confusing for me: I'm non an expert in numpy * but I had assumed that it would do most of those things - vectorize, unroll, etc, either when compiled or through any backend it's using. I understand that numpy's routines are fixed and that mojo might have more flexibility, but for straight up matrix multiplication I'd be very surprised if it's really leaving that much performance on the table. Although I can appreciate that if it depends separately on what BLAS backend has been installed that is a barrier to getting default fast performance.

* For context I do have done some experience experimenting on the gcc/intel compiler options that are available for linear algebra, and even outside of BLAS, compiling with -o3 -ffast-math -funroll-loops etc does a lot of that, and for simple loops as in matrix vector multiplication, compilers can easily vectorize. I'm very curious if there is something I don't know about that will result in a speedup. See e.g. https://gist.github.com/rbitr/3b86154f78a0f0832e8bd171615236... for some basic playing around

[gyrovagueGeist]

Even OpenBLAS (the default iiuc) does all of that and more to optimize for different levels of the cache hierarchy: https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf

I'm not sure where/how they'd be squeezing out more performance unless its better compilation/compatibility with Apple Silicon intrinsics.

Edit: ..Is Mojo using more than 1 core? I'm not sure I understand their syntax and if they are parallel constructs.

Edit2: Yeah Mojo seems to be parallelizing, so the comparison really isn't fair. The np.config posted elsewhere shows that OpenBLAS is only compiled with MAX_THREADS=3 support, and its not clear what their OPENBLAS_NUM_THREADS/OPENMP_NUM_THREADS was set to at runtime.

[andy99]

I'm not super familiar with Mac but I also notice that numpy here is using openblas64. I had thought the go-to was the Accelerate framework? Or is that part of it somehow? If so it would be interesting to see how that impacts performance. Of course it's all kind of an argument for something like Mojo that gives better performance out of the box. Also an argument for why Mojo would be way more interesting if it was open source.

[almostdigital]

Just whatever you get by default with pip install numpy... Changing the benchmark to run with a 1024x1024x1024 matrix instead of a 128x128x128 does speed up numpy significantly though

    Python           119.189 GFLOPS
    Naive:             6.275 GFLOPS            0.05x faster than Python
    Vectorized:       22.259 GFLOPS            0.19x faster than Python
    Parallelized:     50.258 GFLOPS            0.42x faster than Python
    Tiled:            59.692 GFLOPS            0.50x faster than Python
    Unrolled:         62.165 GFLOPS            0.52x faster than Python
    Accumulated:     565.240 GFLOPS            4.74x faster than Python

np.__config__:

    Build Dependencies:
      blas:
        detection method: pkgconfig
        found: true
        include directory: /opt/arm64-builds/include
        lib directory: /opt/arm64-builds/lib
        name: openblas64
        openblas configuration: USE_64BITINT=1 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
          NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SANDYBRIDGE MAX_THREADS=3
        pc file directory: /usr/local/lib/pkgconfig
        version: 0.3.23.dev
      lapack:
        detection method: internal
        found: true
        include directory: unknown
        lib directory: unknown
        name: dep4364960240
        openblas configuration: unknown
        pc file directory: unknown
        version: 1.26.1
    Compilers:
      c:
        commands: cc
        linker: ld64
        name: clang
        version: 14.0.0
      c++:
        commands: c++
        linker: ld64
        name: clang
        version: 14.0.0
      cython:
        commands: cython
        linker: cython
        name: cython
        version: 3.0.3
    Machine Information:
      build:
        cpu: aarch64
        endian: little
        family: aarch64
        system: darwin
      host:
        cpu: aarch64
        endian: little
        family: aarch64
        system: darwin
    Python Information:
      path: /private/var/folders/76/zy5ktkns50v6gt5g8r0sf6sc0000gn/T/cibw-run-27utctq_/cp310-macosx_arm64/build/venv/bin/python
      version: '3.10'
    SIMD Extensions:
      baseline:
      - NEON
      - NEON_FP16
      - NEON_VFPV4
      - ASIMD
      found:
      - ASIMDHP
      not found:
      - ASIMDFHM

[elashri]

If you are looking for improved performance, you will always go with NumPy + vectorization. That's what is important. So I don't know what is the argument here, am I missing something?