[joe_the_user]

Well, if you have an app that's a thin structure on top of GPT-4, adding your own pipe between your client and OpenAI could add a lot to the cost and complexity of the app. Which is to say it's not surprising that people don't do it.

The genius and the craziness of GPT-4 is you can make whole app with a prompt like "now you're a clown painting custom faces on kids based on their favorite animals" and some glue-code. Needing to add a 3 layer network infrastructure with isn't appealing I'd imagine.

[mousetree]

Could be as simple and cost-efficient as a Cloudflare worker that adds your key and passes the query along

[lucakiebel]

Or a 5€/mo Hetzner server running Nginx, though Cloudflares free offering is very generous

[shri_krishna]

You have to use Cloudflare Workers in Unbounded mode to do this (especially if you are streaming using EventSource or WebSockets). Bundled mode won't cut it as it closes connection after 50ms of Javascript execution.

[orf]

That’s 50ms of CPU execution, which does not include waiting for IO.

[jeffy90]

You'd need some kind of authentication as well.

[pxx]

Errrr, I assume your app can pass said authentication? If so, then it's meaningless; that's again a secret not under your control.

[de6u99er]

Not only that, but it makes not much difference if they call the GPT Api directly or through a proxy. Only thing that eould really help is having users register and authenticate through the proxy.

[ceejayoz]

The proxy approach at least lets you rate limit by IP, limit the length of the strings (and thus the token cost), etc. The API key may also grant access to other models, administrative IPs, etc. you don't want people using.

Far, far, far better than nothing.