[omeze]

I think it depends on the use case. The big win IMO is cost savings for developers, tiny models can run on devices so theres no need for inference servers. The real challenge is coming up with a use case that a tiny model can perform. I suspect there’s probably a lot though, like rephrasing or gmail-like suggestions. Theyre tiny features, not full apps, but still seem valuable to me

Fyi me and a friend made a demo app of tinystories we just submitted: https://news.ycombinator.com/item?id=36960333 - from testing it out, anything with <500ms latency and 15 tok/sec feels real-time, and that makes a world of difference in UX

[alexei_rudak]

I mean the case when tiny models can outperform LLM on certain tasks. For example storytelling, and only for Harry Potter world. Something very niche but with maximum quality that can't do other tasks. Just want to see an example of that tiny model.

Did you try to optimise your model for specific tale, game characters etc ?