[rd]

I wonder if I could start a US-based company with good data regulation and just serve open-weight models at a competitive price. I feel like the real barrier is just that most companies willing to adopt AI usage enough to make it worth it at this point don't want to be using inferior models.

[CobrastanJorji]

Here's a free startup idea: operate an open-weight model service, and offer "Verified AI Integrity," which signs the input tokens, the seed for the randomness in selecting outputs, and the model ID, proving that the result of the call to AI was completely "organic" and was not interfered with.

Your main audience would be snake oil salesmen trying to prove their AI products are unbiased and not under the thumb of any outside influence. This doesn't address the biases of the model itself, but that's not your business. Your business is selling tokens and security certificates. If you can get the right angel investor, you could maybe have your new standard required for some government applications.

[tokioyoyo]

Yes, you can. There are multiple inference providers out there. The problem is, it’s hard to beat the Chinese providers in cost. And you also have to compete with frontier model providers’ subsidized offerings.

[dakolli]

They charge the exact same prices. So many people in these comments have no idea what they're talking about. Even if they did charge less, nobody is going to deal with the latency of sending requests to China.

edit: Actually American inference providers are cheaper for Chinese models. There's way more competition here because the Chinese aren't idiots and investing every last dollar they have into data centers for llms that don't make money..

[tokioyoyo]

Can you please link me DeepSeekV4 provider that's cheaper than their official offering? And not all tasks require low latency.

Also, there are a lot of competition in China. Like a lot. You might know better than me as well, but although the biggest AI-labs are based in USA, the adoption is weirdly global. Like as a general sense of what's going on - you can see AI-related ads literally everywhere in Tokyo, almost all the time, in every single screen in public.

[_matthew_]

Cro.ai seems to be: https://crof.ai/

Of course though they are not necessarily a viable solution for companies with security requirements etc. given it is just a single person project, but they still serve as a proof it can be done.

[dakolli]

This costs more.

[gpugreg]

Not as far as I can tell. Are we seeing different things?

For deepseek-v4-pro:

- $0.350 in, $0.003000 cache, $0.80 out https://crof.ai/pricing

- $0.435 in, $0.003625 cache, $0.87 out https://api-docs.deepseek.com/quick_start/pricing

[dakolli]

Deepseek's api platform for V4 Pro is the only example of this, and Deepseek V4 Flash is cheaper (usually) than from Deepseek itself on openrouter via DeepInfra.

Deepseek shot themselves in the foot because they never intended to serve V4 Pro for .80c mm ouput, that was a promotional price that was meant to expire (and still might). They intended for v4 to cost $4.00 per million but Western inference providers drove down the price because they can operate at negative margins to try and push competition out. I can assure you they are losing a ton of money @ ~80cents.

My point is, its Western inference providers that are establishing the floor price of inference. They are willing to operate at a loss in order to put their competition out of business. Chinese providers are typically at or above the prices set by American/western providers if you go looking on the Chinese internet. You aren't going to get deals from China for inference except through this one instance with Deepseek v4 Pro which wasn't even supposed to be permanent pricing.

[RussianCow]

By "cost" I think the parent means the provider's own costs, not the cost of inference to the customer. The cost of land, labor, and electricity are significantly lower in China than in the US.

[mediaman]

There are plenty of US-based inference providers available, including AWS, that serve Chinese models at competitive prices (vs frontier US models). They also have lots of usage. Not necessarily for coding, but for other enterprise tasks.

[dakolli]

Have you heard of openrouter? There's 1000 of these companies already. Do something else.

[fg137]

It's called AWS. Bedrock is right there. Price or data policy is never the issue. The models themselves are the problem -- most large US companies are not going to touch them.

Source: directly involved in these discussions. You can downvote as much as you'd like but you can't ignore the facts.

[RussianCow]

> The models themselves are the problem -- most large US companies are not going to touch them.

Can you expand on this?

[gpugreg]

Some suits with no understanding of how LLMs work are scared that the models might hack them, or believe that they'd have to send data to China because they do not know that open models can be run on your own infra.