[vrighter]

70 somethings per second, is slow. So that means it does take a very significant amount of resources, considering it's running on the same or better hardware. To sustain 70 things per second for thousands of users, it gets expensive really quickly.

[lukev]

My point is that at current API pricing the users are paying enough to cover inference costs.