yeah, but gemma models are shit, objectively speaking. I've been building edge AI tools for my lab group, and gemma lineage hallucinates so much it cannot be used at all. We're almost exclusively using qwen models. Given no image with a prompt for OCR to a gemma model, it'll make things up even if told to null fields not present. Qwen will a) follow instructions (placing things not clearly legible into a dedicated 'notes' field), return null for missing items, and get some pretty wild OCR tasks done quite well. It's almost got the opposite problem, I've had to limit how quickly people can submit an OCR ingested label to the DB because people started trusting it to never make mistakes, while gemma required correction on nearly every scan because of things it made up. So gemma didn't win on tok/s, accuracy, grounding of answers, etc. Theres no conceivable spot where it wins, and this was for qwen-vl:4b vs gemma e4b, so qwen model is 1/3 of the size, runs faster, and is far more reliable. So what does gemma really bring to the table?
It's not the cheapest, its worse than cheaper options, etc. All it really brings is the google label.
I mean, yeah, if it doesn't work at all nobody will use it. But the big players are spending $1k/mo.+ on AI at this point. That's obviously out of reach for many.
What is the target to write for is the key aspect here. Not sure there is enough on the phone for developers to create new experiences. I fear all of them are going to try to automate everything on the phone for the user. Not sure what value that provides. May be I am overly skeptical , let’s see
Maybe I'm a "simple" user but I honestly can't think of anything I'd need automated on my phone that I haven't already automated myself with shortcuts. It's first and foremost a communication device, and I don't need AI automation to reply to my messages or emails for me, nor do I necessarily want purchases/ordering things automated either.
The personal context/search stuff is nice, but that's first party now so yeah, not much room for new experiences.
I run a company of 4 people (including myself) and the "apple tax" (if by that you mean the price premium of macs and iphones vs PC and android) is a tiny fraction (borderline rounding error) of my budget.
Is this more scraping at the bottom of the barrel?
I get it. So many tech companies built their platforms around people submitting their work for sale. Now that things have cooled down they're desperate. This is exactly like what happened to the music and movie industries.
If they want to make money they must take bigger creative risks. AI is the exact opposite of that because it's trained on what's already been done.
I think Apple should do niche AI as in all these companies run LLM on their infrastructure and Apple should focus on running light LLMs on their devices which are capable of doing so.
This means it would be cheap for the end user and they could sell their "privacy" by saying that the user's communication never leaves their devices
Can't wait to try it. Made a small log app for my IBS which allows free text input and uses LLM to create JSON. If that could be done on device with a foundation model, it would be awesome. (and also completely bomb my income model)
I think that can pretty realistically be done with Gemma or Qwen, although maybe with some delay. They run great on android in the Edge Gallery app.
Further, you could allow for voice input by running whisper STT locally, then doing a small context-aware correction pass with Gemma or Qwen to correct words it got wrong.
The issue with those solutions is that it would balloon my app size because I'd need to embed the model, or add a mechanism to download the models afterwards, for something that is essentially a note taking app. But maybe I can make it an option and word it effectively, that is an idea!
Your idea for doing a context aware correction pass on STT is very interesting and something I hadn't thought of yet.
Thank you for your thoughts!
Reading the title I had a very different notion of what to expect. The article has something completely different. I was hoping this would be something for indie app devs.
You can get a certain portion of the population to pay $20/mo. but I think it's a very small population who's paying enough to actually cover frontier models in agentic workflows right now.
Either I've fallen in with a unique group of non-techy people willing to pay for an LLM subscription, or you just not giving enough credit to it. I guess time will tell
> Only about 3% of households were paying for AI in February, using the most recent numbers available from the Bank of America Institute, which researches consumer trends based on the bank's customer transactions.
But even among these people I doubt most spring for the $100 plans, let alone are willing to pay hundreds or thousands of dollars per user the way corporate users do.
The funniest part of all of this is that, I as a techy dev type person, pay $0 for any LLM account. No, I'm not cheating with a paid by employer account. I just don't use it. So I guess my little group is breaking all of the stereotypes