|
|
|
|
|
|
by stymaar
17 days ago
|
|
|
The catch is that the token generation speed is going to be limited by network latency, making it unbearably slow to run over the internet. It can be great on a local network though, especially if your workload is prefill-heavy (more text input to process than output tokens to emit). |
|
|