[jon-wood]

Everything you do a risk/reward equation, you just don't usually see it drawn out quite so starkly. Getting out of bed in the morning carries a risk that you'll trip and crack your head on the floor. Crossing a road carries a risk of being hit by a bus. Eating food carries a risk of choking on it. The same is true in computer security. The only truly secure computer is one you don't turn on, and even that carries some risk of an attacker breaking in and stealing the storage from it.

Whether you agree that the potential harms outweigh the benefits in this case or not those calculations are always happening, so yes, I guess you're right. That is society in a nutshell.

[vrganj]

But if you eat food, I don't risk choking. They want us to take the risk for their reward.

[pixl97]

But if I drive a car, You do run the risk of getting ran over. We can come up with any number of analogies of varying rightness and wrongness here.

[daveshistory]

And then there's a whole truckload of case law about liability that comes into play.

We haven't yet written those laws for "AI."

[hombre_fatal]

What do you have in mind?

You're paying for their services to collect reward for yourself, but also deciding your own risk/reward when choosing e.g. how much access to grant Claude for any given task.

I guess there's the case where the more capable Claude is, the more someone else can use it to find vulns in your services while Anthropic collects their subscription money? But that is mitigable risk that you shipped regardless of what Anthropic is doing.

[6gvONxR4sf7o]

My point wasn’t about risk vs reward, or in their words “harm” vs reward. It’s about how increasing the opportunity for reward increases the justifiable harm. “X is bad (unless it makes me rich).”

I guess it’s the fact that Anthropic usually frame this around morality and risk to society that makes it different. Instead of “risk/harm to me vs reward to me,” their framing reads as “risk/harm to us vs reward to me” or “immorality vs reward to me.” That’s what makes it feel like a great metaphor.

The standard cost benefit analysis we all do justifies increasing the harm to others if the opportunity to benefit ourselves goes up.