|
|
|
|
|
|
by HarHarVeryFunny
3 days ago
|
|
|
1000 tokens per sec is still massively slower than serving a normal web page - if something doesn't respond in a few seconds many people give up. I'm not saying there aren't any use cases for super-fast (and super-expensive) generation, but it does seem a bit niche. If it was free then sure faster is better, but what are the mainstream use cases where people might pay 3x more for a faster version of something that is already fast? I think it would have to be an application where it paid for itself - where the 10x faster response was actually worth more than 3x the cost to you - where the extra speed was worth the extra cost. |
|
|