Hacker News new | ask | show | jobs
by biduskamil 3 days ago
Local is the way. Any benchmarks on latency it has on CPU?
1 comments
unusual_typo 3 days ago
I just ran the benchmark on my macbook. 582 ms for 1k tokens and 4.64 s for 8k
link