[war321]

Even if it's just open weights and not "true" open source, I'll still give Meta the appreciation of being one of the few big AI companies actually committed to open models. In an ecosystem where groups like Anthropic and OpenAI keep hemming and hawing about safety and the necessity of closed AI systems "for our sake", they stand out among the rest.

[loceng]

To me it will be most interesting to see who attempts to manipulate the models by stuffing them with content, essentially adding "duplicate" content such as via tautology, in order to make it have added-misallocated weight; which I don't think an AI model will automatically be able to determine, unless it was truly intelligent, instead it would require to be trained by competent humans.

And so the models that have mechanisms for curating and preventing such misapplied weighting, and then the organizations and individuals who accurately create adjustments to the models, will in the end be the winners - where truth has been more honed for.

[meowtimemania]

Why would openai/anthropic's approach be more safe? Are people able to remove all the guard rails on the llama models?

[Der_Einzige]

Yes, it's trivial to remove guardrails from any open access model:

https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in...

https://huggingface.co/failspy/Llama-3-70B-Instruct-ablitera...

[Zuiii]

Humanity is so fortunate this "guardrails" mentality didn't catch on when we started publishing books. While too close for comfort, we got twice lucky that computing wasn't hampered by this mentality either.

This time, humanity narrowly averted complete disaster thanks to the huge efforts and resources of a small number of people.

I wonder if we are witnessing humanity's the end of open knowledge and compute (at least until we pass through a neo dark ages and reach the next age of enlightenment).

Whether it'll be due to profit or control, it looks like humanity is posed to get fucked.

[alfalfasprout]

They're not safer. The claim is that OpenAI will enforce guard rails and take steps to ensure model outputs and prompts are responsible... but only a fool would take them at their word.

[seoulmetro]

Yeah.. and Facebook said they would enforce censorship on their platforms to ensure content safety.. that didn't turn out so well. Now it just censors anything remotely controversial, such as World War 2 historical facts or even just slightly offensive wording.

[Spivak]

You're really just arguing about the tuning. I get that it's annoying as a user but as a moderator going into it with the mentality that any post is expendable and bringing down the banhammer on everything near the line keeps things civil. HN does that too with the no flame-bait rule.

[seoulmetro]

HN moderation is quite poor and very subjective. The guidelines are not the site rules, the rules are made up on the spot.

HN censors too. Facebook just does it automatically on a huge scale with no reasoning behind each censor.

Censorship is just tuning people or things you don't want out. Censorship of your own content as a user is extremely annoying and Facebook's censorhsip is quite unethical. It doesn't help safety of the users, it helps safety of the business.

Also Facebook censors things that are not objectively not offensive in lots of instances. YouTube too. Safety for their brand.

[koolala]

Censorship isn't moderation.

[zelphirkalt]

The banhammer can quickly become a tool of net negative though, when actual facts are being repressed/censored.