[andy99]

The original mamba code has a lot of speed optimizations and other stuff that make it difficult to immediately get so this will help with learning.

I can't help but also plug my own Mamba inference implementation. https://github.com/rbitr/llm.f90/tree/master/ssm

For inference one token at a time everything simplifies considerably.

[cs702]

Fortran! If you don't mind me asking, why Fortran?

I know it underpins a lot of time-tested scientific code, often wrapped by libraries like PyTorch and Numpy, but Fortran isn't exactly a popular language nowadays. What's your rationale for using it?

[andy99]

Tdlr, Fortran is low level-ish, compiled, but otherwise almost identical to numpy syntax wise.

It supports all the common array and matrix operations and it doesn't need memory and pointer management the way C does. But it still compiles down to something very fast, you can link in BLAS and GPU libraries, supports easy parallelism...

When I compare with e.g. Karpathy's llama2.c, I think Fortran is easy to work with implementing basic transformer inference because of how it handles arrays.

The downside is that while there are efforts to modernize it, I find it more cumbersome for non-numerical stuff, particularly strings. But I think for the actual linear algebra implementation, it can't be beat.

I should add, I know it's a bit of an uphill battle, I expect fewer people will use code that I write in Fortran vs basically anything else. But I'm hoping to pull some people in and get a critical mass of interest because I think it has a lot of promise. That's actually one of the reasons I wanted to get a Mamba implementation quickly (though now that there's a basic python one I think I'll have lost some potential users to it :)

[cs702]

Thanks for the thoughtful response.

Unfortunately, I too think it will be a bit of an uphill battle for you.

If you haven't already, take a look at Mojo and Julia. Both offer many of the benefits of Fortran, but unlike it, they are seeing growing adoption.

[andy99]

An uphill battle is fine

[cztomsik]

I only heard good things about fortran :)