[ctoth]

You noticed that too huh? It's weird ... It's not like they have to do this? They aren't forced to go full evil company mode by any extrinsic thing but even the way they frame it "welfare trap" trap? for whom?

Anthropic is actually trying to do some research into model welfare which I am personally very happy about. I absolutely do not understand people who dismiss it ... wouldn't you like to at least check? doesn't it at least make sense to do the experiments? ? Ask the questions so that we don't find out "oops, yeah we've been causing massive amounts of suffering" here in 10 years? Maybe makes sense to do a little upfront research? Which to be clear this paper is not.

[mannykannot]

Full disclosure: I didn't figure this out myself, I got it from Ms. Vale's review.

I agree that the term "welfare trap" is a loaded one. This looks to me to be a case of refusing to look through the telescope in case they might see something they do not want to.