Hacker News new | ask | show | jobs
by bloody-crow 899 days ago
This might be the best reason to consider a Pro model next time I'm upgrading my iPhone.
1 comments

iPhone 15 and iPhone 14 Pro, 14 Pro Max have exactly the same CPU and amount of RAM (Apple A16 Bionic and 6GB). This is also true for iPhone 14 and iPhone 13 Pro, Pro Max (Apple A15 Bionic and also 6GB).
I don't play games or do anything too resource-demanding on my phone normally. Pro models typically have more memory than non-pro models and running LLMs on device might be the only scenario where it can realistically make a difference for me.
Smaller 3B LLMs (like phi-2) work fine on newer non pro models, at full context lengths. Running 7B models on even 8GB iPhone 15 Pro and Pro Max phones involves reducing the context lengths to 1k or fewer tokens, because the full context length KV cache won't fit on these devices.