[a2128]

I wonder at what point does increasing LLM throughput only start to serve negative uses of AI. This is already 2 orders of magnitude faster than humans can read. Are there any significant legitimate uses beyond just spamming AI-generated SEO articles and fake Amazon books more quickly and cheaply?

[Workaccount2]

The way things are going it looks like tokens/s is going to play a big role. O1 preview devours tokens and now Anthropic computer use is devouring them too. Video generation is extremely token heavy too.

It sort of is starting to look like you can linearly boost utility by exponentially scaling token usage per query. If so we might see companies slowing on scaling parameters and instead focusing on scaling token usage.

[adwn]

How about just serving more clients in parallel? I don't see why human reading-speed should pose any kind of upper bound.

And then there are use cases like OpenAI's o1, where most tokens aren't even generated for the benefit of a human, but as input for itself.