[goodcjw2]

This is pretty cool to have more generic GPU-based support.

Might also be related: Tensorflow Lite Core ML delegate enables running TFLite model on CoreML enabled iOS devices.

https://www.tensorflow.org/lite/performance/coreml_delegate

Would nice to see performance comparisons on M1 Mac/iPad for which way is more performance and efficient. (Admittedly TF vs TF-lite is a 100% apple to apple comparison, pun-intended).

[b9a2cab5]

Just running native models on OpenCL has existed for a while now, through PlaidML and other companies. Unfortunately you need CUDA support for custom kernels for a lot of modern architectures like Transformers, and AMD's CUDA transpiler leaves a lot to be desired.