[PoignardAzur]

It's funny, just two hours ago there was a thread by a pundit arguing that these AI advances don't actually give the companies producing them a competitive moat, because it's actually very easy for other models to "catch up" once you can use the API to produce lots of training examples.

Almost every answer in the thread was "this guy isn't that smart, this is obvious, everybody knew that", even though comments like the above are commonplace.

FWIW I agree with the "no competitive moat" perspective. OpenAI even released open-source benchmarks, and is collecting open-source prompts. There are efforts like Open-Assistant to create independent open-source prompt databases. Competitors will catch up in a matter of years.

[dTal]

Years? There are already competitors. I just spent all evening playing with Claude (https://poe.com/claude) and it's better than davinci-003.

To be fair it is easy to radically underestimate the rate of progress in this space. Last Wednesday I conservatively opined to a friend "in 10 years we'll all be running these things on our phones". Given that LLaMA was running on a phone a few days later, I may have been a little underoptimistic...

[hackernewds]

how do you run LLaMa on a phone?

[glenneroo]

It's "all" over the news now ;) https://arstechnica.com/information-technology/2023/03/you-c...

Here's results of running on Android: https://github.com/ggerganov/llama.cpp/issues/124

This is about running llama on a Raspberry Pi: https://github.com/ggerganov/llama.cpp/issues/58

...and this is where people have been posting their results running on all sorts of hardware, though I don't see anything Android related: https://github.com/facebookresearch/llama/issues/79

Obviously the larger models won't run on such limited hardware (yet) but one of the next big projects (that I can see) being worked on is converting the models to be 3bit (currently 8bit and 4bit are popular) which cuts down required resources drastically with minimal noticeable loss in quality.

I think starting with FlexGen barely 4 weeks ago, there have been some pretty crazy LLM projects/forks popping up on github almost daily. With FlexGen I felt like I was still able to stay up-to-date but I'm getting close to giving up trying as things are moving exponentially faster... you know it's crazy when a ton of noobs who have never heard of conda are getting this stuff running (sometimes coming in flexgen discord or posting github issues to get help, though even those are becoming rarer as one-click-installer's are becoming a thing for some popular ML tools, such as oobabooga's amazing webui tool which has managed to integrate almost all the hottest new feature forks fairly quickly: https://github.com/oobabooga/text-generation-webui

I just helped someone recently get oobabooga running which has a --listen option to open the webui to your network, now he's running llama on his tablet (via his PC).

[karmasimida]

It could take about a year or so.

But I think you should forget about self-hosting at this point, the game is up.