[hyferg]

You can return log probs per token generated. This can be used to asses the confidence the model has in handling tasks which involve nominal data.

If that’s not helpful, were you getting at having the model return some rich data about the attention weights that went into generating some token?

[jsemrau]

For most of our models we return more information. Especially if you look at it from a vendor/customer perspective I believe this to be quite important.