Hacker News new | ask | show | jobs
by gojomo 1765 days ago
How large of a model download would be required to remove the dependency on a cloud service?
1 comments
freeone3000 1765 days ago
No more than 1GB, and inference would be dependent on the speed but no more than 10 seconds. Completely reasonable as a standalone program.
link
Siira 1765 days ago
Is there a guide on running it locally? I am currently using some classic algorithms to summarize my RSS feeds, but they aren’t very good.
link
freeone3000 1764 days ago
Sure! Here's the weights https://huggingface.co/sshleifer/distilbart-cnn-12-6 , here's the model https://huggingface.co/transformers/model_doc/bart.html?#tra... . Uses standard pytorch loading, so write your python program around that and it should be fine.
link
lrem 1765 days ago
Would that require running a separate server locally? That's all kinds of non-techie hostile :(
link