[gmiller123456]

"Scalability" and "Single Board Computer" don't really belong in the same sentence. That said, today you can get a refurbished mini PC with a lot more power, for a lot less money than the higher end SBCs. But I didn't see any info on how portable this project is to other hardware.

[m00x]

I think the biggest advantage here is that you can run it on the GPU using shared memory, which I'm not sure how widespread it is on mini PCs (at least not intel NUCs).

You could run it using OpenVINO on IntelCPUs, but the performance would probably take a hit. It would be a lot easier though since you can just use ggml.