[cyberlogika]

I'd like to think someday AI will help us make those technical decisions, even though we won't fully understand them because the human brain simply can't process the multi-dimensional complexity that a computer theoretically handles no problem. That leaves a lot of room for fear-mongering, but one man's utopia is another man's dystopia, I suppose.

As long as humans benefit, I say it's a worthwhile goal to at least explore. If that leads to the extinction of the human race through genetic defect or similar existential tragedy stemming from this, then maybe we just weren't cut out for this gig and should go the way of the dodo. Maybe on another planet, a higher intelligence will figure out how to peacefully coexist with an intelligence of their own creation/modification. Maybe this is just all an inevitable aspect of the evolutionary algorithm at work; who are we to think we can avoid it?

[noobermin]

I've seen a number of articles that there is suspicion that AI (ie., ML'd coefficients in a set of matrices) just reinforces already held prejudices in certain cases which do not actually jive with the facts. How are we to know the AI won't make decisions based on current prejudices that end up also dooming mankind in the future?

[cyberlogika]

And those are definitely valid concerns. And if those 'bugs' aren't preventable or fixed, I guess the question then becomes: whose irrationality kills everyone first, human's or machine's? Perhaps the situation is inevitable, just part of the Great Filter that decides which intelligence-type survives the birthing process into post-evolution. If machines don't threaten our very survival, something (or someone) else will.

Perhaps prejudices may be unavoidable in any intelligence, since we build stereotypes as predictive models and ML makes similar abstractions and assumptions which influence perception and predictions. Tangentially related, when TBI patients whose emotional centers are impaired, so is their ability to make decisions [1]. Building a decision-making network without an emotional center may be impossible, since the two seem to be naturally correlated. It's probable an AI won't ever truly be 'emotional,' at least not in the near term.

So, in the end, it's reasonable to assume that the AI of the future that can crunch these complex problems won't tell people the answers, they will merely provide possible solutions with varying likelihood of success depending on goals and constraints. In the end, we will have to decide our own fate, and my point is maybe we don't have control over deciding our fate either way--such is the nature of fate.

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3032808/