AFAIU NPUs are for things like voice input/output, computer vision/hand gesture io, knowing how many people/who are in front of the camera etc. Always on, real-time "ai peripherals" - not content generation.
I believe Microsoft calls them "SLMs - Small Language Models".
Of those who do, I can see students and researchers benefiting from small models. Students in particular are famously short on money for fancy hardware.
My experience trying one of the Phi models (I think 3, might have been 2) was brief, because it failed so hard: my first test was to ask for a single page web app Tetris clone, and not only was the first half the output simply doing that task wrong, the second half was a sudden sharp turn into python code to train an ML model — it didn't even delimit the transition, one line javascript, the next python.
> My experience trying one of the Phi models (I think 3, might have been 2) was brief
The Phi models are tiny LMs, maybe SLM is more fitting label than LLM (Large -> Small). As such, you cannot throw even semi-complicated problems at them. Things like "autocomplete" and other simpler things are the use cases you'd use it for, not "code this game for me", you'll need something much more powerful for that.
> Things like "autocomplete" and other simpler things are the use cases you'd use it for, not "code this game for me", you'll need something much more powerful for that.
Indeed, clearly.
However, it was tuned for chat, and people kept telling me it was competitive with the OpenAI models for coding.
Asking a leading LLM to "code a game" is a tall order. I have found a lot of success with self hosted small models to accomplish coding that would have taking me months without. I just break down the "code me a game" to its parts.
I believe Microsoft calls them "SLMs - Small Language Models".