Hacker News new | ask | show | jobs
by altcognito 53 days ago
Billions upon billions going to these companies.

25k reward from a selected group of people if you help us determine whether or not someone can use our tool to generate weapons of mass destruction.

4 comments

It's worse than that, for partial successes they encourage people to submit the attempt but reserve the right to not pay anything (they may, at their discretion, give a partial reward if they feel like it).
That's pretty much how every bounty works... obviously it's going to be at their discretion for an incomplete attempt.
it's unusual to have to sign the NDA for a rejected bounty
No it isn't. Confidentiality terms are the norm.
Because it can't and it's a publicity stunt. It achieves three goals:

1) Underscores to the general public that the models are amazingly powerful and if you're not using them, your competitors will out-innovate you,

2) Sends the message to regulators that they don't need to do anything because the companies are diligent to prevent harm,

3) Sends the message to regulators that they sure should be regulating "open-source" models, because these hippies are not doing rigorous safety testing.

Both Anthropic and OpenAI have been playing that game for years.

If it can’t, then it makes more sense to make the bounty as high as possible instead of a measly $25k
If it's an existential threat to humanity, and if OpenAI is valued at nearly $1T, why set the bounty at a measly $25k? The going rate for an iPhone zero-day is six to seven figures. Some companies will pay you more than $25k for a website XSS.

Because this is not a serious effort to address a serious risk. It's a PR stunt, the bounty is for a simple jailbreak and not a bioweapon, and they don't necessarily want to spend a lot of money or get people really invested in breaking their safety filters.

They don't want anyone to actually do it.
I’m glad people are starting to recognize this, but when will the general public? Never?
They're probably expecting that it can be done without too much effort so they just want to see all the unique ways people are doing it.
They’re probably expecting biological weapons of mass destruction can be created without too much effort, so are curious to see all the nifty ways people can create biological weapons of mass destruction?
I was talking about bypassing the ChatGPT safeguards, that's what this bug hunt is about.
Though it could be a Honeypot they are probably hoping to train on all the ways someone might try to do this. Or maybe funds are really low and they need a smoke screen for a really bad actor to go in and try to do it for real.