[torben-friis]

I don't think you got my point. I'm not criticizing anthropic for deciding to engage with the pope, I'm pointing at the state we're at where a for profit company is doing individually the work of understanding how their disruptive tech should fit in the wider world.

Saving distances, it's like Glock engaging with spiritual leaders to figure out when it's ethical to kill. This should not be their area of decision, and if it starts being so there is clearly a giant gap for the entities that should be leading this instead.

[losvedir]

I think Glock sounds silly because it's a relatively small weapon and company. But I wouldn't think it at all strange if you used, say, Oppenheimer engaging with the pope around nuclear weapons at the start of the nuclear age. Or maybe even better Enrico Fermi when he was developing nuclear reactors, since those had both the potential for cheap, clean energy and the spectre of nuclear weapons, which feels more analogous to the state of AI right now to me.

[torben-friis]

Those are actually wonderful examples to my point, because it was the state leading the manhattan project. It is the role of the state to engage with issues such as how to handle the new discovery in terms of national security, constitutional rights, etc.

I would even argue that, for such a disrupting discovery, some sort of international approach should be considered - everything from monoply risks through energy usage to economical risks if massive unemployment is possible. We instead have private interests dealing with (or ignoring) these issues by themselves.

It's something that already misfired with the advent of social networks, where many of the current issues could have been avoided if the state had actually showed up to engage with the problem.