Hacker News new | ask | show | jobs
by dTal 1193 days ago
Years? There are already competitors. I just spent all evening playing with Claude (https://poe.com/claude) and it's better than davinci-003.

To be fair it is easy to radically underestimate the rate of progress in this space. Last Wednesday I conservatively opined to a friend "in 10 years we'll all be running these things on our phones". Given that LLaMA was running on a phone a few days later, I may have been a little underoptimistic...
1 comments
hackernewds 1193 days ago
how do you run LLaMa on a phone?
link
glenneroo 1193 days ago
It's "all" over the news now ;) https://arstechnica.com/information-technology/2023/03/you-c...

Here's results of running on Android: https://github.com/ggerganov/llama.cpp/issues/124

This is about running llama on a Raspberry Pi: https://github.com/ggerganov/llama.cpp/issues/58

...and this is where people have been posting their results running on all sorts of hardware, though I don't see anything Android related: https://github.com/facebookresearch/llama/issues/79

Obviously the larger models won't run on such limited hardware (yet) but one of the next big projects (that I can see) being worked on is converting the models to be 3bit (currently 8bit and 4bit are popular) which cuts down required resources drastically with minimal noticeable loss in quality.

I think starting with FlexGen barely 4 weeks ago, there have been some pretty crazy LLM projects/forks popping up on github almost daily. With FlexGen I felt like I was still able to stay up-to-date but I'm getting close to giving up trying as things are moving exponentially faster... you know it's crazy when a ton of noobs who have never heard of conda are getting this stuff running (sometimes coming in flexgen discord or posting github issues to get help, though even those are becoming rarer as one-click-installer's are becoming a thing for some popular ML tools, such as oobabooga's amazing webui tool which has managed to integrate almost all the hottest new feature forks fairly quickly: https://github.com/oobabooga/text-generation-webui

I just helped someone recently get oobabooga running which has a --listen option to open the webui to your network, now he's running llama on his tablet (via his PC).

link