The metal support for pytorch is a real sh*t show at this point - all APIs are either legacy or unfinished. Also, app memory is about 3GB even for the largest phones.
It really still feels that it‘s all in its infancies.
Apple already has their own environment in CoreML and the neural engine. They have ways to train/run models on it in highly optimized formats and unified memory.
PyTorch came from PCs running on GPUs with their own pool of dedicated memory and a slower link to main memory.
I know people have been speeding it up a lot on Apple processors, but I’m not sure it’s a good indicator of how ready the iPhone is for running models for end user programs.
PyTorch came from PCs running on GPUs with their own pool of dedicated memory and a slower link to main memory.
I know people have been speeding it up a lot on Apple processors, but I’m not sure it’s a good indicator of how ready the iPhone is for running models for end user programs.