|
|
|
|
|
by RussianCow
16 days ago
|
|
Once you've used a model that runs at hundreds of TPS, it's hard to go back. Everything completes so quickly that you can iterate without breaking out of flow state. My biggest gripe with slow (<50tps) LLMs is that I've lost all the mental context I built up by the time it's done, which makes it extremely difficult to explore or iterate on solutions. |
|
Working with models that are super fast, but slightly dumber (like mimo-v2.5-pro-ultraspeed) is amazing, I feel like I'm still the one that's actually making every decision.