Hacker News new | ask | show | jobs
by glintik 813 days ago
How much RAM browser wants to run LLM local processing?
3 comments
dchest 813 days ago
Usually around 5 GB for a 7B 4-bit quantized model.
link
skeeter2020 813 days ago
probably less than it needs for a few dozen open tabs based on my past profiling experiences...
link
kevindamm 813 days ago
How long are you willing to wait?
link