[mattlondon]

Is this really "on" the RPi though? They mention inference on ARM Mali GPU (which is cool! But I don't think the RPis use Mali right?) but it wasn't clear to me if this was just using the RPi to take a picture and send it to an AWS server to do the actual inference, then return the results?

The last time I fiddled with inference on a RPi was several years ago when it took a couple seconds even for inference of a basic ImageNet CNN reading low-res images from the RPi camera (can't remember specifics sorry - I think it was on a RPi 2 or 3). FPS was something like 0.5-0.2FPS which made "real time" usage kinda difficult. I don't know what performance would be like now on a RPi4, but GPU accelerated inference on the Broadcom GPU would be nice.

[simbaz]

Hi, Mattlondon, it's really nice to have your reply. You are asking a good question, the 'Mali GPU' supporting is not for RPi, it's on the board which has 'Mali GPU' as RK3288/3399 or others (MTK/....). For now, the reference code is running on RPi cpu, a light MXNET model do the extracting of face feature, the detection code is also running on RPi, the classifier is on RPi too. The API server is the APP logic server, AWS/MINIO is storage layer. The light model is https://github.com/deepinsight/insightface/wiki/Model-Zoo#34...

which running MobileNet as backbone. IF running on Mali GPU, ResNet50 can be used. Inference duration depends on the GPU power. if running on Mali 720 MP2, should be something close to 0.3s.