[tskj]

I think this is decisively the wrong way to think about it. Yes, layering hypotheticals like that means that any one scenario is extremely unlikely to be the thing that gets you, but that doesn't mean the shape of the problem is wrong.

It's like arguing with someone who doesn't believe in using seat belts when driving. "Why should I put them on?" they say, and when you try to explain what might go wrong they won't listen to any explanation that isn't a hyper-concrete hypothetical. So finally you give in and say, "Well, when we get onto the highway, a truck might lose control and hit us", and their response is "I don't think that's very likely, it seems highly improbable that today we will be hit by a truck when getting on the highway".

I agree with OP that this seems like the kind of thing where the unknown unknowns are so great that the correct approach is serious caution, and that any demand to know exactly how or why it will go wrong, falls in the trap where every specific example is very unlikely to be the thing that goes wrong, but still in total there's like an 80% chance that it goes horribly wrong. I don't know if we have the terminology to talk about this kind of failure mode. "You shouldn't play God" maybe? At least you shouldn't ask for specific examples of how things could go wrong, if you're going to turn around and claim each one highly improbable.

[xi_studio]

"I don't know if we have the terminology to talk about this kind of failure mode."

We actually have and is called RISK.

RISK = Probability * Damage.

Applied to the seatbelt event we have a death level damage and a high probability of happening given recent studies, so using a simple belt could easily save you from deadly accidents.

Applied to any unrealistic scenario we have insane level damages but also an incredibly low probability (near 0) so RISK = ~0%

[tskj]

Well that's the point about unknown unknowns though, we actually have no idea what the probability is. But we do know it's not low, that's the only unlikely scenario.

We only know it's a very complicated system with many interlocking dependencies, and based on what we know about complicated systems in general, as well as biology in particular, is that if any one of these unknown dependencies break, the whole system can fail catastrophically.

Therefore the probability that something will go wrong is very high, and the damage could easily be irreparable damage to the species, if not extinction. Does that not give an intolerably high risk?

[throwaway173738]

I don’t think this technology we’ll get people to stop having sex to the degree that IVG is all we’ll ever use. And to the degree that we believe some humans are genetically destined for anything, that’s the line of thinking that leads to dystopia.

[tskj]

I don't know man, people basically aren't having sex already. The fertility rate is way below replacement in all modernized countries, half of gen Z has never had sex at all, contraception is free, people struggle to get pregnant the more chronically ill the population gets, and the people who actively do want to get pregnant are going to want to use whatever technology there is to improve things.

[soco]

But outside the bible, who says we must be more and above replacement rates? Is such a tragedy for humanity if there are only 4 billion humans in 100 years? 4 happier billions? It doesn't HAVE to go up all the time, and between extinction (zero humans) and overcrowding (X billions) there are many unknown ways to go.

[tskj]

Unfortunately most of society and the economy is built on the foundational assumption of a growing (or at least stable) population. There will be some extremely tough challenges to solve with an actively declining population.

[throwaway173738]

A lot of these issues have more to do with how we’ve constructed our society than with some loss of purity. If you want people to have more sex, make cities and towns more livable, get people out of cars, give them living wages so they have free time, create societal supports for childcare, make giving birth and pediatric care completely free, and support people forming associations outside of work. These are all barriers that have appeared basically my lifetime and it’s way more likely that people are making informed choices than that there’s some horrible transformation we’re all undergoing.

[nme01]

This framework would be hard to apply to unknown unknowns. That's why in software engineering you'd apply canarying. Then the longer timeframe for potential negative effects, the slower the adoption of a new process should be.

[soco]

The history of humanity, I say of hominids even, was defined by humans playing it unsafe - migrating, sailing, inventing bombs, you name it. We played god before even we invented gods, and reached this point in time. Should we say "this is best we can do, let's stop everything"? Nah, not likely.

[vintermann]

"To play God" isn't just to do something spectacular which wasn't possible before. It's to use other people - to put yourself on a higher teleological level than other people, declaring their purpose (whatever it is) as fully subordinate to your purpose (whatever it is).

As if they were a tool you created. Obviously, if I carve a piece of wood into a spoon, it's I who get to say that what used to be a piece of wood now has a purpose of moving soup to mouths. The carved piece of wood now has a purpose, but it's wholly subordinate to my purpose (whatever it is).

You don't have to actually design people to play God - you can subordinate others to your purpose without doing that, that's what various God-kings in history did. But it certainly gives you a head start if you make them.

You can make something and don't subordinate it to your purpose. In our culture, we see children that way. We claim, basically, that we didn't deliberately design a child, we only obeyed our own natures without really having much choice in the matter, and thus we and our child is on the same teleological level.

This is not a cultural universal. In many cultures, parents (in particular fathers) would say basically "I made you, so you must do as I please, you have no reason for existing except for my purposes". It was a hard won battle for our culture to assert that children matter for their own sake and not just for their parent's.

Many things in this thread makes me want to say "Y'all MFs need Jesus". I won't say that, but I will say that you should stop and think about why it is that the Catholic church is so difficult about contraception, why Christians in general have historically made such a big deal of the difference "born of God" vs "Created by God" (arianism, etc.), and what that story of Abraham and Isaac was really about. Whether you agree or not, there's much about other people you will never understand if you don't think about teleological levels.

[tskj]

Thanks, that's an interesting analysis. I'm not a Christian, but I definitely get the sense that we're playing with an extreme danger here, and that we're not being sufficiently humble / cautious / in awe of the sacred -- and that such hubris might literally lead to human extinction, or close to it.