[martindbp]

Yes, nobody is claiming it's inevitable based on nothing, it's based on first principles thinking: economics, incentives, game theory, human psychology. Trying to recast this in terms of "predestination" gives me strong wordcel vibes.

[bonoboTP]

It's a bit like pattern matching the Cold War fears of a nuclear exchange and nuclear winter to the flood myths or apocalyptic narratives across the ages, and hence dismissing it as "ah, seen this kind of talk before", totally ignoring that Hiroshima and Nagasaki actually happened, later tests actually happened, etc.

It's indeed a symptom of working in an environment where everything is just discourse about discourse, and prestige is given to some surprising novel packaging or merger of narratives, and all that is produced is words that argue with other words, and it's all about criticizing how one author undermines some other author too much or not enough and so on.

From that point of view, sure, nothing new under the sun.

It's all too well to complain about the boy crying wolf, but when you see the pack of wolves entering the village, it's no longer just about words.

Now, anyone is of course free to dispute the empirical arguments, but I see many very self-satisfied prestigious thinkers who think they don't have to stoop so low as to actually look at models and how people use them in reality, it can all just be dismissed based on ick factors and name calling like "slop".

Few are saying that these things are eschatological inevitabilities. They are saying that there are incentive gradients that point in a certain direction and it cannot be moved out from that groove without massive and fragile coordination, due to game theoretical reasonings, given a certain material state of the world right now out there, outside the page of the "text".

[keiferski]

I think you’re missing the point of the blog post and the point of my grandparent comment, which is that there is a pervasive attitude amongst technologists that “it’s just gonna happen anyway and therefore whether I work on something negative for the world or not makes no difference, and therefore I have no role as an ethical agent.” It’s a way to avoid responsibility and freedom.

We are not discussing the likelihood of some particular scenario based on models and numbers and statistics and predictions by Very Smart Important People.

[theSherwood]

I agree that "very likely" is not "inevitable". It is possible for the advance of AI to stop, but difficult. I agree that doesn't absolve people of responsibility for what they do. But I disagree with the comparison to religious predestination.

[bonoboTP]

I'm not sure how common that is... I'd guess most who work on it think that there's a positive future with LLMs also. I mean they likely wouldn't say "I work on something negative for the world".

[keiferski]

I think the vast majority of people are there because it’s interesting work and they’re being paid exceptionally well. That’s the extent to which 95/100 of employees engage with the ethics of their work.

[welferkj]

Nobody serious is claiming theological predesination is based on "nothing", either. Talk about poor pattern matching.

[theSherwood]

You are, of course, entitled to your religious convictions. But to most people outside of your religious community, the evidence for some specific theological claim (such as predestination) looks an awful lot like "nothing". In contrast, claims about the trajectory of AI (whether you agree with the claims or not) are based on easily-verifiable, public knowledge about the recent history of AI development.

[welferkj]

It is not a "specific theological claim" either, rather a school of theological discourse. You're literally doing free-form association now and pretending to have novel insights into centuries of work on the issue.

[theSherwood]

I'm not pretending to any novel insights. Most of us who don't have much use for theology are generally unimpressed by its discourse. Not novel at all. And the "centuries of work" without concrete developments that exist outside of the minds of those invested in the discourse is one reason why many of us are unimpressed. In contrast, AI development is resulting in concrete changes that are easily verified by anyone and on much shorter time scales.

[bonoboTP]

Relatedly, it would be bordering on impossible to convince Iran about the validity of Augustine, Aquinas or Calvin, but it was fairly easy with nuclear physics. Theology isn't "based on nothing", but the convincing power of the quantum physics books happens to be radically different from Summa Theologiae, even if both are just books written by educated people based on a lot of thought and prior work.