|
|
|
|
|
by subarctic
17 days ago
|
|
I haven't seen it do it by itself that much so far. Even then, a lot of the time is spent waiting for some shell command to complete, which isn't inference. I guess being able to do parallel inference is like having multiple cores in a cpu, you can get by without it by just sharing that single core, and having it just gives you a performance boost under certain circumstances |
|
And yes, I agree. I find the experience better less for speed and more for context management. But it's far from necessary.