[3pt14159]

> despite this disadvantage some applications just cannot afford the load of every single request touching the DB or cache.

Disagree. This is one of the simplest things alive to distribute. Split the query from the token do the query in the DB and the token lookup is effectively a distributed hash table lookup (assuming the token is, say, a UUID). Once the DB query comes back store the result pending the successful retrieval of the token.

What's difficult is handling something like millions of concurrent video downloads / uploads - not looking up tiny tokens.

[paulddraper]

tiny tokens, but needed for every single operation

[3pt14159]

Sure, but the really hot tokens could be cached right next to the DB. Plus how many operations is a person doing per second? If its more than a couple you can batch them pretty easily.