Hacker News new | ask | show | jobs
by Jack5500 1200 days ago
This seems like a great first step to a truly open source LLM
2 comments

Indeed. All the weights for all the models will be available one way or the other very soon.

The proprietary nature of the weights is not going to be a bottleneck for more than a month, if I had to guess.

The other bottle-neck to personal use — the hardware required to run (not train from scratch) the thing - is going to be gone within the year I bet. I would assume some clever bloke is going to be able to prune the model or decrease the precision of the weights and discover you can get good-enough results with 1/10th of the memory.

What happens then?

Well, probably some Very Bad Things.

If we can confirm the 13B model is as good as GPT3 then at least we have a target for reproduction without FB restrictions.