[never_inline]

> Something I think would be interesting for model APIs and consumer apps to exposed would be the probability of each individual token generated.

Isn't that what logprobs is?