[mythz]

Claude 3.5 Sonnet still holds the LLM crown for code which I'll use when wanting to check the output of the best LLM, however my Continue Dev, Aider and Claude Dev plugins are currently configured to use DeepSeek Coder V2 236B (and local ollama DeepSeek Coder V2 for tab completions) as it offers the best value at $0.14M/$0.28M which sits just below Claude 3.5 Sonnet on Aider's leaderboard [1] whilst being 43x cheaper.

[1] https://aider.chat/docs/leaderboards/

[dsp_person]

DeepSeek sounds really good, but the terms/privacy policy look a bit sketch (e.g. grant full license to use/reproduce inputs and outputs). Is there anywhere feasible to spin up the 240B model for a similarly cheap price in private?

The following quotes from a reddit comment here https://www.reddit.com/r/LocalLLaMA/comments/1dkgjqg/comment...

> under International Data Transfers (in the Privacy Policy): """ The personal information we collect from you may be stored on a server located outside of the country where you live. We store the information we collect in secure servers located in the People's Republic of China . """

> under How We Share Your Information > Our Corporate Group (in the Privacy Policy): """ The Services are supported by certain entities within our corporate group. These entities process Information You Provide, and Automatically Collected Information for us, as necessary to provide certain functions, such as storage, content delivery, security, research and development, analytics, customer and technical support, and content moderation. """

> under How We Use Your Information (in the Privacy Policy): """ Carry out data analysis, research and investigations, and test the Services to ensure its stability and security; """

> under 4.Intellectual Property (in the Terms): """ 4.3 By using our Services, you hereby grant us an unconditional, irrevocable, non-exclusive, royalty-free, sublicensable, transferable, perpetual and worldwide licence, to the extent permitted by local law, to reproduce, use, modify your Inputs and Outputs in connection with the provision of the Services. """

[mythz]

It's a 236B MoE model with only 21B active parameters that ollama is reporting having 258k downloads [1] (for 16/236 combined) whilst Hugging Face says was downloaded 37k times last month [2], which can run at 25 tok/s on a single M2 Ultra [3].

At $0.14M/$0.28M it's a no brainier to use their APIs. I understand some people would have privacy concerns and would want to avoid their APIs, although I personally spend all my time contributing to publicly available OSS code bases so I'm happy for any OSS LLM to use any of our code bases to improve their LLM and hopefully also improving the generated code for anyone using our libraries.

Since many LLM orgs are looking to build proprietary moats around their LLMs to maintain their artificially high prices, I'll personally make an effort to use the best OSS LLMs available first (i.e. from DeepSeek, Meta, Qwen or Mistral AI) since they're bringing down the cost of LLMs and aiming to render the technology a commodity.

[1] https://ollama.com/library/deepseek-coder-v2

[2] https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-In...

[3] https://x.com/awnihannun/status/1814045712512090281

[bilekas]

> """ The personal information we collect from you may be stored on a server located outside of the country where you live. We store the information we collect in secure servers located in the People's Republic of China . """

Is that even legal in regards to EU users ?

[onli]

Of course not.

[yard2010]

Well, good luck prosecuting Winnie the Pooh

[bilekas]

Their services will just be blocked in the EU instead.. we've seen it in Italy early on with ChatGPT..

[samstave]

When will we have token-flow-aware-networking gear... Surely NVIDIA and others are already doing special traffic shaping for tokenFlows?

Whats the current state of such tech/thought/standards/vendors?

[yumraj]

There’s no company info on DeepSeek’s website. Looking at the above, and considering that, it seems very sketchy indeed.

Maybe OK for trying out stuff, a big no no for real work.

[redeyedtreefrog]

The names of their researchers are on this recent paper: https://arxiv.org/pdf/2408.15664

Their terms of service say "The DeepSeek Open Platform is jointly owned and operated by Hangzhou DeepSeek Artificial Intelligence Co., Ltd., Beijing DeepSeek Artificial Intelligence Co., Ltd. "

And they're funded by https://www.high-flyer.cn/en/fund/ which the FT did an article on: https://www.ft.com/content/357f3c68-b866-4c2e-b678-0d075051a...

In terms of the personal data you share when using their models, I can't see why they would be any more or less nefarious than big Western tech companies.

That said, if you're using a model based in China then by providing them with data and feedback you are in a very small way helping researchers in China catch up with/keep up with/overtake researchers in the West. Maybe in the long term that could end badly. And if you are concerned about the environment, it's entirely possible their training and inference is run using coal power stations.

[rfoo]

> There’s no company info on DeepSeek’s website.

It's backed solely by a hedge fund who do not want to draw attention to their business. So yeah, as sketchy as DESRES.

[dotancohen]

Might be good for contributing to open source projects. But not for clients' projects.

[stavros]

I'm making a small calendar renderer for e-ink screens (https://github.com/skorokithakis/calumny) which Claude basically wrote all of, so I figured I'd try DeepSeek. I had it add a small circle to the left of the "current day" line, which it added fine, but it couldn't solve the problem of the circle not being shown over another element. It tried and tried, to no avail, until I switched to Claude, which fixed the problem immediately.

43x cheaper is good, but my time is also worth money, and it unfortunately doesn't bode well for me that it's stumped by the first problem I throw at it.

[jadbox]

What's the better plug-in among Continue Dev, Aider and Claude Dev?

[phren0logy]

You probably already know that Aider is not a plugin, but just in case - it's a program that runs from the terminal. I think the results are very impressive, and it readily handles multiples source files for context.

[FloatArtifact]

What kind of hardware is required for the local llm for the continue stack?

[jodrellblank]

Uh, what price format is "14 million cents per 28 million cents"?

[sabbaticaldev]

14 cents per million input/28 cents per million output tokens

[jodrellblank]

Oh right, thanks.