[nixpulvis]

This is a serious problem with the fact that it's nearly impossible to understand what a "token" is and how to tame their use in a principled way.

It's like if cars didn't advertise MPG, but instead something that could change randomly.

[uoaei]

Like if cars measured fuel efficiency or range using the knobs in the tread on your tire.

[amitprasad]

Relevant post: https://modal.com/blog/dollars-per-token-considered-harmful

(disclaimer: I work with the author)

[nixpulvis]

I completely agree that requests are what should be charged for. But I think there are two things, given that requests aren't all going to cost the same amount:

1. Estimate free invoicing the requests and letting users figure it out after the fact. 2. Somehow estimating cost and telling users how much a request will cost.

We have 1, we want 2.

[claw-el]

Also, certain models are more verbose than the others. We are basically at the mercy of a model who likes to ramble a lot.

[konfusinomicon]

im fiarly certain the knob on the machine that controls length of redundant comments and docblocks is cranked to 11. it makes me curious how much of their bottom line is driven by redundant comment output.