Hacker News new | ask | show | jobs
by lambdaba 1271 days ago
Will it ever be an option, perhaps the first option, given some evolution in the software and/or hardware?
1 comments

Ever? Likely. Hardware keeps improving, and better training techniques will almost certainly keep shrinking model sizes ceteris paribus. But one should also remember that these are static models that can't learn anything that was not present in the original training corpus, so for some use cases that rely on current information they're simply not a good match. And training a model like this requires vastly more hardware (and human) resources than just using it. Never mind the issue of collecting a corpus in the first place.