[jeanloolz]

Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation, and this is in a way an AI disaster we should be worrisome. Power corrupts, great power greatly corrupts, this is not entirely new.

From where I stand, it looks like the pandora box has already been opened anyway. The era of hugginface and/or Llama2 models is only going to grow from there.

[seydor]

Let's not be so hasty. I m glad to see that MistralAI is signing this letter. their small model is truly amazing, the most useful open source model i ve tried so far. But this can change tomorrow, they may be deemed illegal and disappear from public view, it has happened before. So i'm downloading and saving whatever model i think is great.

[mwint]

Genuine question - does it seem plausible that a few GB of content could truly be wiped off the Internet?

I’m not in the piracy scene, but my impression was they routinely pass full res movies around the Internet without much barrier to discovering and downloading them, at least to technically competent users. Is that still true?

[PumpkinSpice]

This is really not about an engineer keeping a bootleg model in your basement. It's about the barrier for entry for commercial products. Or the ability to curate improved open-source implementations in the long haul, for that matter - as past a certain scale, this entails creating a non-profit of some sort to pay your bills.

Plus, while it's definitely the case that with sustained interest, old data tends to linger around... the moment the interest wanes, it's gone. I've been on the internet for a while and there are so many hobby sites, forums, and software projects from the early days that are simply gone for good (and not on archive.org).

[sillysaurusx]

The Pile was. It’s still available but no one will touch it, mostly due to books3.

The difference is that a few people with lots of resources take on legal risk. In the piracy example many people with few resources take on risk, which works out since no one wants to sue people with no money.

[kenan_warren]

The Pile is still used to train LLMs and it's still very much available on the net. I agree it's a risk to train your models on the dataset until the legal implications are worked out, but it doesn't seem to be stopping people.

[superkuh]

The purpose of regulations like these are not to prevent a thing from happening. They're so that normal behavior is criminalized but not enforced unless you happen to rock the boat some day.

[seydor]

the old models will become deprecated if they are not upgraded, and won't incorporate new information. Even if the files are available they will become abandonware.

[trentnix]

> Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation

Well yes, that’s precisely why they are lobbying for it.

[blacksmith_tb]

That does seem awfully convenient, yes. But I am also skeptical that Meta is railing against regulation out of their commitment to protect scrappy little startups doing AI...

[Taek]

The gap between Meta and other proprietary ai ecosystems (OpenAI, Google, Anthropic) is pretty large. Meta necessarily is only as far behind as open source as a whole is behind.

If open source catches up, Meta can ride the tailwind and also catch up. I'm sure Meta will flip its position once it doesn't feel outclassed by the competition.

[rightbyte]

I honestly don't see the cynical motive I know is there in what Facebook is doing, releasing Llama. I am down to that they just want to screw with Microsoft for buying "OpenAI", which is fine by me.

[rhizome]

>Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation

It's a homesteading land grab, plain, simple and pure.

There is a competitive landscape, first-mover advantages, incumbent effects, et al that are being anchored in e.g. Sam Altman's interests and desires at this very moment. If you want a vision of the future of garage AI, imagine a boot stomping on Preston Tucker's face over and over. The current AI industry's goal is to preserve an image of openness and benefit while getting ready to pull the ladder up when the time is right.

[shrimpx]

Power is already concentrated in the sense that only the wealthy can train. We should make kickstarter-type models to fund open source AI training, because it'll likely continue to cost millions to tens of millions for cutting edge models.

With Llama2 we're at Meta's mercy, as it cost 20M to train. No guarantee Meta will continue to give us next-gen models. And even if it does, we're stuck with their training biases, at least to some extent. (I know you can fine-tune etc.)

[flockonus]

While i don't disagree, that's missing the forest for the trees - the point is having the freedom to share models and continue to open-source.

I'll argue that between stable diffusion and llama 2, there is nothing highly specific that prevents [very] large amount of people from adopting these models and specializing for them own needs.

The tragedy would be if those went away.

[shrimpx]

Yeah but llama2-level AI may be insignificant in power compared to future models, which may be inaccessible to the public. Even assuming the algorithms/code are open, people at large won't be able to create working models.

[flockonus]

Not sure at all, when you look at amount of opensource being done in regards to quantization. One of many examples: https://huggingface.co/TheBloke/Llama-2-7B-GGML

[autoexec]

> Power is already concentrated in the sense that only the wealthy can train.

That situation will change as technology evolves. We'll eventually reach a point where a normal desktop PC can train AI. The wealthy will always be able to do it faster, but the gap will shrink with time.

The trick is making sure that laws aren't put in place now that would restrict our ability to do that freely once the technology is there, and that massive data sets are compiled and preserved where the public has free access to them.

[CatWChainsaw]

Normal desktop PCs are going to be neutered to subscription-based terminals that you can take home with you rather than go to a central location to access.

[shrimpx]

That didn't happen with cloud though. I know it's not entirely apples to apples, but you still need a computer the size of a large building to serve Netflix, etc.

[lelandbatey]

Only if you're trying to serve netflix-quality stuff to hundreds of thousands of people. If you're trying to replicate "Netflix the product" (live video streaming with a slick interface) to a small set of individuals, you can do that with a personal computer (see Jellyfin, Plex).

Comparing Netflix (and most highly profitable computer businesses) to the world of producing AI models by training is not going to be fruitful. Netflix takes a lot of effort to operate but you can do on the small scale what Netflix does, quite directly. You can't replicate an AI model like ChatGPT-4 very easily unless you have all the data and huge compute that OpenAI does. Now, once the model has been produced, you can operate that model on the small scale with maybe less amazing results (see llama.cpp, etc) but producing the model is a scale problem like producing high quality steel. You can't escape the need for scale (without some serious technological developments first).

[autoexec]

Companies will always have an advantage with scale. It's not like you need a super computer though. You can have a single desktop media server in your home that does what netflix does without any problem. A single media server can even serve multiple homes.

Netflix cheats. They send non-supercomputer boxes out to ISPs to install locally. If I could convince every ISP to install a bunch of my media servers people could watch my shows from anywhere in the US too.

[shrimpx]

I guess we're speculating that training llama2 will drop by 1000x or something, so anyone can train their own llama2 from scratch for about $2k.

I don't think compute cost has dropped by 1000x since 20 years ago. Maybe by 10 to 50x. And if you add in the demand for higher quality, the cost has probably increased. Like encoding a video for streaming 20 years ago, at that standard, may have cost roughly the same as it does today, or more, when you factor in the increases in resolution and quality.

My prediction is that training the latest model will continue to cost millions to tens of millions for a long time, and these costs may even increase dramatically if significantly more powerful models require proportional increase in training compute.

Unless of course we have some insane algorithmic breakthrough where we find an AI algorithm that blows llama2 out of the water for a small fraction of the compute.

[PeterStuer]

What compute was available 20 years ago that would be the equivalent of an H100?

[jmfldn]

True but there is a really difficult trade off here between a big tech monopoly and decentralised open source anarchy. I say this as a Linux-using FOSS lover, but we need to open this up as much as possible without allowing superpowers to escape into the wrong hands. I guess strong regulation is the key but I'm just handwaving here really. I don't know how the tradeoff should be made. I see great dangers in hyper-powerful tech being in the hands of the few or indeed of everyone.

By the way, I'm not sure how easy it will be to stop bad actors since barriers to entry are exponentially lower to developing a malicious AI tool than, say, developing a nuke.

[pdonis]

> without allowing superpowers to escape into the wrong hands

The wrong hands will have the same access to whatever "superpowers" AI gives regardless of what regulations are or are not put in place. Regulations can't and won't stop potential bad actors with state-level resources, like China, from using any technology they decide they want to use. So trying to regulate on that basis is a fool's errand.

The real question is, what will put the good actors in a better position to fight the bad actors if it ever comes to that: a big tech monopoly or decentralized open source anarchy? The answer should be obvious. No monopoly is going to out-innovate decentralized open source.

> I'm not sure how easy it will be to stop bad actors since barriers to entry are exponentially lower to developing a malicious AI tool than, say, developing a nuke.

Since some bad actors already have nukes, the answer to this should be obvious too: it's what I said above about the wrong hands getting access to technology.

[ta1243]

China, Iran, North Korea, Iran, US, Israel, Europe aren't bad actors. The bad actors don't have state level resources.

[dragonwriter]

> China, Iran, North Korea, Iran, US, Israel, Europe aren't bad actors

On a very good day, as many as three of those might simultaneously not be bad actors.

They aren't bad actors whose access to AI technology is likely to be meaningfully impacted by regulation (but, for certain of the non-US ones, that hasn't stopped the US from trying before), but that's a different issue.

[pdonis]

> China, Iran, North Korea, Iran, US, Israel, Europe aren't bad actors.

Seriously? You don't think China, Iran, and North Korea are bad actors? What planet are you on?

[ta1243]

Well depends on your point of view

But I don't see China or North Korea firing nukes or even blowing up western buildings. They are limited by the threat of response.

A rogue wacko in his basement can make Kim Jong Un look like Theodore Roosevelt.

[pbronez]

The term you’re looking for is “constrained” and “unconstrained” actors. The Chinese and North Korean governments are state level actors, constrained by their internal institutions and power dynamics. A rogue wacko is unconstrained - they do whatever they want with no limitations beyond their own mind and individual capabilities.

Examples:

In the Iran Hostage Crisis you had a constrained actor (Iranian government) making somewhat rational choices to use hostage taking as a negotiation tactic.

In the Oklahoma City Bombing, you had unconstrained actors (Timothy McVeigh and Terry Nichols) blowing up a building with a vehicle borne improvised explosive device for personal reasons.

[PeterisP]

We do see North Korea, Russia and Iran funding cyberattacks (e.g. major ransomware operations) and supplying weapons to various external groups. If/when we get to a stage when a wacko in his basement possessing some AI tool could be dangerous, we can expect state-level bad actors to make and deliver such tools to any wacko they'll consider likely to use against the west i.e. us.

[jmfldn]

Interesting point. Perhaps you're right. I hope so, as someone who otherwise believes in the FOSS ideal.

[slowmovintarget]

The fallacy you're falling for is that this is "hyper-powerful tech."

All of the AI danger propaganda being spread (see [1], for example) has the purpose of regulatory capture. You could have said all the same things about PageRank if it had come out in 2020. A malicious AI tool is harder to assemble than straight up cracking. The people who can do it are highly-trained professional criminals taking in millions of dollars. Those people aren't going to be stopped because the source is closed. (I'm thinking of that criminal enterprise based in Israel that could manipulate elections, blackmail any politician anywhere in the world... etc. They were using ML tools two years ago to do this.)

The ML tools are already in the wrong hands. The already powerful are trying to create a "moat" for themselves. We need these models and weights to spread far and wide because the people who can't run them will become the have-nots.

[1] https://news.ycombinator.com/item?id=38117930

[Vetch]

Let's assume that these LLMs are very useful and provide boosts to their users. It follows that anyone not leveraging them where applicable would be at an economic disadvantage. Without opensource models, everyone would have to pay a Google or Microsoft tax for the pleasure of using a service which could not exist without our work. All code, writing, art, data would have to be continually sent to their servers for anyone wanting to leverage their tools. You might use a FOSS editor or shell but you are still sending all your data to Microsoft servers.

The ones in control of the models also control what sentences are sanctioned, this is a problem the more widely LLMs are used. To add insult to injury, while we are not allowed private use of the models, governments and ad-tech surveillance capabilities will skyrocket.

Do you see the problem here? The capabilities of opensource models are not anywhere near high enough to justify such a cost, now or anytime soon.

And it won't end there. As the march of progress continues, we will see the AI doom crowd agitate for tighter surveillance of money flows, limits on private compute, bandwidth limits to homes, tracking what programs we run on our computers, on who is allowed to read the latest in semiconductor research and on and on.

[dragonwriter]

> but we need to open this up as much as possible without allowing superpowers to escape into the wrong hands

There are no superpowers, and the wrong hands are the ones least effected by any effort at restricting distribution by “strong regulation”.

[najork]

I completely agree. AI naturally lends itself to monopoly formation, and stifled competition in the space is something that we as consumers (and stakeholders of human progress) should be wary of. I wrote down my thoughts in a brief blog post and I'd love your take on it.

https://radiantai.com/blog/building-resilience-into-ai

[skybrian]

I disagree entirely. We shouldn't be hoping for a complex ecosystem of small, shady companies nobody ever heard of, like happened with ad networks [1]. Or worse, a scam-filled ecosystem like cryptocurrency. Or worse, an underground ecosystem like the criminals who share cracked passwords and credit card numbers.

Big companies are easier to regulate.

[1] https://lumapartners.com/lumascapes/

[pdonis]

> Big companies are easier to regulate.

But the problem isn't regulating the big companies, or the smaller companies, or underground entities. The problem is state-level adversaries like China who might misuse a technology, whether it's AI or anything else. Such adversaries can't be regulated by laws or executive orders or UN declarations; they have proven that many times in the past. The only way to control them is to have sufficient counter-capability against whatever assets they have. And government regulation is a terrible way to try to achieve that goal.

[skybrian]

State-level actors are one problem. People defrauding the elderly are a different problem. Ransomware is a third problem. They are all problems.

The idea that there's only one important problem is a fallacy.

[pdonis]

> The idea that there's only one important problem is a fallacy.

I have made no such claim.

The people advocating for regulating AI are claiming it will solve all the relevant problems--i.e., that it will prevent AI from doing great harm. So pointing out a problem that the regulations will not solve is refuting the claims the advocates of regulation are making. That was my point.

[skybrian]

That's surprising and seems like an overreach. Whatever those advocates claimed, it doesn't seem very relevant to deciding with or not a particular regulation is a good idea.

[pdonis]

> Whatever those advocates claimed, it doesn't seem very relevant to deciding with or not a particular regulation is a good idea.

I don't see why not. The whole point of regulations is to regulate, i.e., to keep the regulated activity within some particular bounds. If the regulation won't accomplish that, then it is pointless. Unless, of course, the actual purpose of the regulation is not the same as the purpose that is publicly stated--which is exactly what happens with regulatory capture.

[smokel]

One could also try to be friendly towards strangers, and not have them be their adversaries in the first place.

[pdonis]

> One could also try to be friendly towards strangers, and not have them be their adversaries in the first place.

We have tried this with China, going back to Nixon opening up trade relations in the early 1970s. It hasn't helped.

[smcleod]

Really? You think Amazon, Google, Apple, Facebook etc…. Are well regulated?

The US has shown time and time again it’s complete incompetence when it comes to meaningful regulation of large companies.