[seiji]

strong AI can be immortal

My grandfather was very racist and never changed as long as he lived. I'm glad most people aren't immortal. There's no guarantee a full AI wouldn't be tempted by evil and just become republican (or worse, a VC).

It can also recursively improve itself.

So can people (the more you know, the more you can learn), but most don't. It's important to remember "intelligence" isn't an abstract concept—intelligence is also embodied in personality—and personalities have wishes and goals and desires and loves and hates and that one song they can't get out of their head. A true "strong AI" will be fully conscious, not just algorithmic function bating.

Good luck telling a mildly strong godform to stop tripping on youtube videos and instead solve the global economic stability equation over lunch.

This advantage will lower the bar for strong AI even more.

That's kinda foofy conjecture. Being good at rectangular grid outcomes isn't necessarily a step in any direction towards a hands-off tax evaluating robot.

It feels really really good to talk about how AI will be a hundred billion trillion times smarter than the combined brainpower of all humans that have ever lived, but it feels good in the same way thinking dead people live again after they die feels good—it triggers that warm wishful thinking parietal lobe that removes a bit of reason for the sake of an overarching calmness.

Enthusiasm is great, but tempering with real expectations and less technopriesthood is better.

[badminton1]

You can recruit neurons within your own brain, not plug the equivalent of 4 brains into your brain and increase your cognitive abilities.

Now, while many people die and their ideas die with them, there are built in aspects of our brain that are hardwired. People eat animals because they are tasty, people follow the life of Kim Kardashian and waste money buying a rolex and wear fur just because the stupid way our social functions are hardwired in our brain.

[quietplatypus]

Pretty much this.

[nzonbi]

In the grand scheme of things, what is the point of humans repeating billion of times the same cycle?: born, grow, learn, achieve some things, have some fun, and die. It is awesome a few times. But it is worth to repeat and repeat that for ever? Not if there is a much better option: The singularity. It has the potential to enhance the human race in formidable ways. Yes,there is the risk that the AI may chose to destroy us. But in my opinion is a bet that we should take. Just try our best to not be destroyed in the way. But we shouldn't leave this opportunity unexplored. It would be the biggest achievement of humanity.

[hutzlibu]

It is NOT the same cycle. Every life is different. So what the people learned 500 years ago made sense for them - but they would face a hard time now. Our knowledge of daily living will be mostly obsolete in 200 (or much less) years, too.

Fresh minds don't have the burden of outdated ideas... quoting Max Planck: "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."

[armitron]

You are assigning value to "us" beyond what might actually be there. What if our only evolutionary value is to give rise to machine intelligence [that will go on to explore the universe] and die off?

We are primitive forms of life after all and are at the point where we should be able to say, with certainty, that machine intelligence will be objectively superior and something that we should feel OBLIGATED to give rise to, regardless of consequences.

[nzonbi]

> What if our only evolutionary value is to give rise to machine intelligence

There is no prescribed script about what our evolutionary value should be. We will be whatever we ourselves make of us. Unless something unexpected happens -more advanced aliens intervening with our specie for example- Our human instinct is to grow, explore, discover, compete, share as much as we can. AI should be seen as a tool to our goals. Not as an end by itself. There is of course the possibility that we may loose control to the AI. That is not our desired goal, but it could happen. In that case our future will not be our decision. We will be at the mercy of whatever the AI choose for us. That will depend on what kind of AI we built, and how the rampant AI chooses to modify itself. But we will strive to reach our human goals. Because that is the human nature.

[armitron]

Just because you or me see no prescribed script, doesn't mean it's not there. You make a philosophical point here, but my point is more than that.

"We will be whatever we ourselves make of us" sounds meaningless to me. The universe obeys certain physical laws, evolution is directed and follows a specific pattern. We have up to this point been bound by both. Occam's razor says this will continue to be the case.

I think we are already at the crossroads where we can more than speculate about the function of machine intelligence in evolutionary terms, but hopefully sooner rather than later we will have actual _evidence_ that we can look at and use to refine our assumptions.

[tim333]

Yeah, the dying thing's a bitch.