This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
> Just because they published it doesn't mean that others haven't already discovered it.
That’s irrelevant for the moral evaluation. Everyone is responsible for their own actions. If you choose not to kill anyone it won’t stop others from killing. But the fact that this is so doesn’t give you a license to kill.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”