Hacker News new | ask | show | jobs
by yawnxyz 1202 days ago
Wow that's so incredible. Thanks for putting this together!

Do you have any machine specs associated with this? Can an old-ish Macbook Pro run this service?

I'm also curious, since I'm new to all this — is it possible to run something like this on Fly.io or does it take up way too much space?

1 comments

7B is the default. If it's quantized to 4 bits, that's a 3.9 GB file.