Hacker News new | ask | show | jobs
by spacebanana7 1133 days ago
AFAIK some azure services can be run on premises if you're a big enough customer. Don't think GPT APIs are available yet but might in theory be possible in the future. Self hosting a model the size of GPT4 would be insanely difficult and expensive. but might be a workable solution for data sensitive enterprises like Apple, JPM and government orgs.
3 comments
jefftk 1133 days ago
That would be very dangerous for OpenAI to agree to: high risk of it all getting leaked.

I can't think of any enterprise software you might run on premises that has leaking incentives anywhere near as high.

link
coredog64 1133 days ago
Microsoft has had two different iterations of on-prem Azure. The TL;DR is that you’d buy a full rack of hardware from them and you could manage it using Azure front end.

The first iteration was a miserable failure. The second was a heavy lift to install: Previous employer was an Azure customer and actually bought the hardware. It then took months to get it installed and working, at which point the appetite for that capability was gone.

link
whimsicalism 1133 days ago
I can practically guarantee you GPT will never be available through such a scheme.
link
spacebanana7 1133 days ago
Why? I’m genuinely curious.

How is running GPT on Prem any different than other proprietary software like Oracle DBs or even Windows server.

When it gets leaked, they can use licences/copyright to prevent anyone from using it to compete with them.

link
whimsicalism 1133 days ago
1. The value in proprietary software largely lies in the source code, which is not available in on-premises, you are talking about compiled binaries. There is no such thing as "compiled binaries" for neural network weights.

2. License/copyright - Even in the best possible case, there are plenty of jurisdictions that don't care about US copyright and would love to get a hand on those weights.

3. We haven't seen copyright cases around model weights before. If I managed to exfiltrate OpenAIs model weights, I would continue training for a few iterations and then I think it would be quite difficult to prove that I actually have the same model as OpenAI. This is untested, why would they risk it.

4. Running these models requires a ton of resources, vastly beyond the typical onprem deployment - why would Azure invest in making this possible when it really could only impact a very small percentage of companies?

The weights are OAIs lifeblood, I imagine they are very protective of them.

link