[takeiteasyy]

This will require a cpp port first to run on e.g. apple silicon?

[wsgeorge]

No, but to benefit from the ggml ecosystem, yes. Someone's taking a stab at it: https://github.com/nikisalli/falcon.cpp

[rovr138]

This is using huggingface. It's python. There's no reason it wouldn't run on arm.

https://huggingface.co/tiiuae/falcon-40b

I'll also add that the fact something is in C++ doesn't mean it will run on arm or that it can be compiled in it.