[michaelmrose]

The first thing that comes to mind is a question. If we made something as smart as you why would you be legally allowed to use it as a slave?

Next the logical argument is that technology doesn't stand still. If we start with a human level AI we can gain increased runtime simply by throwing more compute at the problem. There is no particular reason we have to run it in real time why not 10x or 100x or 1000x.

What would you produce in your field if we gave you a century or a millennium between now and next year?

This leaves the last and perhaps obvious point that an AI that can improve itself can use these centuries of run time to work on such improvements.

If you have human level AI tomorrow you don't have merely human level AI for long and once something surpasses you, you are poorly positioned to predict or control it. This is not to say it would inherently be malicious but does your pet cat get much of a vote in the running of your day to day life?

[browsergap]

I think this is wrong (not to say you are wrong but this idea which is very common).

The idea that we develop AGI and it is this program that is what we run on a regular device, and then we can make it 10x smarter effectively by running it on 10 devices or 10x as fast. I think more likely will be AGI will be achieved first on the biggest teraflop supercomputers that we have, and it will for a long time be the app that takes a lot to run. And probably the first AGI will not be quite as smart as a human, but basically we will have no other reference point for what it is as smart as so we will call it that.

Also, I think there will be some sort of non-linearity effects that mean that you can't just "scale up" intelligence by adding more processors. It will work to a point, but then the curve flattens. Consider that the global total IQ is already approx 800 billion, but our planet is still pretty dumb. I mean this to also apply to scaling a "single" AGI up in speed. Linear speed gains will have diminishing returns I think.

Also, I speak about the productization and allocation of it. It will not be this "come one come all" "gather round" everyone can partake sort of thing. It will be a product, like night vision or GPS, and the secret government military uses will get the best quality, and the rest of us will get smarter shopping.

Further, if it really is linearly scalable, then it certainly will be controlled. It will be more controlled than enriched uranium in that case, and even if not so scalable is still going to be very controlled if it is at all transformative.

I think the various technological, political and commercial realities will distinctly flatten/soften/smooth the predicted "singularity" discontinuity blast wave into a humdrum speed bump that appears to most of humanity as a better iPhone (basically).

This is pure speculation. We shall see.

[michaelmrose]

I didn't suppose that it would run on your laptop. What is possible on top end supercomputers scales up as well as technology improves.

In regards to scaling why can't we use more compute to run the a single ai faster and faster instead of simulating more AI especially if its running on a supercomputer with a high bandwidth communication between nodes?

If between point A and point B you have 100 times the compute why don't you effectively simulate a century of thought for your single AI instead of simulating 100 AIs.

[browsergap]

I think there will be some limit that means it's not about speed. I think we can't just scale up the speed factor (even if we reach speeds that are fast enough for this in the first place).

Maybe it's related to experience/embodiment. What's a 100 years of thought if you don't have the experience, OODA feedback loop to inform it?

But maybe that's not it, and it's related to something else. I just think the simplistic idea that "once we have it, we only need to make it faster" will not work for some reason. I don't think creating ASI will be that easy. It's basically creating a god. I think if you sped up how an average human thinks, they don't become a god.

Consider psychological trauma and issues. Something that happened 20, 30, 40 years ago, people are still obsessed with and scarred by today. That's not adaptive, and in many ways, that's highly stupid. But it's so common in intelligent humans. If we could have compressed that 40 years of thought (and even experience) to 1 second, they still have made no progress wrt that factor. It's not just about speed, it's about the quality or nature of thought.

But I'm not saying that analogy explains it. I just think there will be some reason why simply adding speed will not some spectacular revolution make.

Another, but still too glib way of saying it is, say we create an AI equivalent to a human. Still humans are pretty dumb, all things considered. Say we speed it up. Now we just have a AI that's more quick to be dumb than a regular human. Do you know what I'm saying?