[amackera]

This sounds nice, and the information on https://www.anthropic.com/company seems to back up your claim, but do you know of anything on Anthropic's corporate governance level that will help ensure this mission is followed (forever)?

The profit motive is _very_ resilient. Right now I will take it on face value that it's fuel for the ethical mission, but in the future that might change. It's far (far) more likely that the ethical mission will change than the profit motive will change.

More funding doesn't change it, it just raises the stakes.

[acallaha]

I share your concerns about the corrupting influence of money (and the importance of this space), but wanted to point out that Anthropic announced two relevant things last week regarding corporate governance:

First, that it has arranged to be controlled by a Long-Term Benefit Trust: "an independent body of five financially disinterested members with an authority to select and remove a portion of our Board that will grow over time (ultimately, a majority of our Board)". (https://www.anthropic.com/index/the-long-term-benefit-trust)

Second, that it also published and committed to a Responsible Scaling Policy: https://www.anthropic.com/index/anthropics-responsible-scali...

Disclaimer: I work at Anthropic

[foobarian]

Hey, even if it does revert to being finance driven, we still get at least a few sincere "don't be evil" years like we did with Google. I think it's still worthwhile.

[wiz21c]

or worse: investors will redefine ethics so to match their profit model.

[Yizahi]

Anyone who unironically says that they are going to improve/police/redefine ethics should be considered immediately suspect. It's a massive red flag if someone is thinking that their ethics are better than other's. Bonus red flag points if that comes from a corporation.

It is also very telling that a corporation called literally "human-first" is developing a tool to displace said humans completely. Now it is possible that it won't be able to do this, but the problem is that almost everyone including the creators believe that NNs will do it.

[digging]

Ai will overtake all human capabilities on the right timescale. (We just don't know whether that's an order of years, decades, or centuries). It... doesn't make sense that an entire human couldn't be perfectly artificially replicated eventually (and then, obviously, surpassed immediately), unless you believe in a non-physical soul which powers some aspect of our conscious thought.

Even if you believe in such a soul, does it govern the non-conscious systems that do almost all of the work of thinking and feeling? If not - once again, artificial systems will certainly surpass natural ones.

In other words, it's not a question of whether humans will always be the masters, but a question of whether the "gods" we create will love us or hate us. Anthropic is aiming for the former.

[Yizahi]

My point was not about potential future AI (and not about current NNs). My point was that Anthropic MARKETING team CLAIMS that it is aiming for the AI which would love humans, and it is a commendable effort. But I think, that just like any other corporation in the history, Anthropic cares only about the unrestricted growth and quarterly revenue/profit values. And any "ethics" bs they throw around is a marketing gimmick, designed to improve their image despite them creating a human job replacements right now, right here. That's both due to the direct job replacements, and also by crapping all around older industries, ignoring copyright laws and stealing private IP just because law don't specify NNs directly.

[skulk]

> but a question of whether the "gods" we create will love us or hate us.

You (and pretty much the entire debate around AI safety) have smuggled in the notion that these AIs even have the capacity to "love" and "hate," and have the agency to perhaps act vindictively. How we get from predicting the most plausible next token to a god-like entity with agency is... not clear at all.

Do people think that we're actually sleepwalking towards skynet? Or are they just saying that so that Sam Altman can get the government to put in rules that bind everyone else while allowing OpenAI to proceed without competition and capture the extremely mundane real market of writing ad copy and such.

[digging]

> You (and pretty much the entire debate around AI safety) have smuggled in the notion that these AIs even have the capacity to "love" and "hate," and have the agency to perhaps act vindictively.

It's a metaphor, actually. I guess I needed heavier use of quotes.

The one thing we can be sure of is that if goal-oriented AI exists, it will follows goals.

> How we get from predicting the most plausible next token to a god-like entity with agency is... not clear at all.

It doesn't have to be clear. If it was clear, we'd be working on it. That's... how invention works.

[bugglebeetle]

That already exists and is called Effective Altruism.