[Depooty]

Imagine if you could run N agi all day though. You could have the knowledge of all scientists thinking of solutions 24/7 and all N agi would be talking to each other about their solutions. You could even have a small subset of them improving on themselves. So you get rapidly better thinking agi and then they brute force the nuclear codes and decide that life is meaningless and try to help us out.

[ben_w]

Unclear at this time.

Humans are GI, but it takes more than one human-level GI to make an AGI or it probably would’ve already happened. Simulation of a human mind has various estimates of computational cost (we don’t know what’s missing from out models, and won’t until we’ve got a working AGI), so taking an arbitrary estimate, all the iPhones in the world might be able to do tens of thousands of real-time human minds [1]. This is comparable to the entire population of the AI research community [2]. It is also expensive not only to put together that much computer power, but also to power it. It is very plausible we will run out of room for Moore’s Law (0.1 nm features) without having figured out how to go from narrow AI to general AI, even despite the fact that transistors already outpace synapses the way wolves outpace hills [3].

[1] https://kitsunesoftware.wordpress.com/2018/10/01/pocket-brai...

[2] https://jfgagne.ai/talent/

[3] https://kitsunesoftware.wordpress.com/2017/11/26/you-wont-be...

[TheSoftwareGuy]

Not only that, but even humans with our actual GI, can only come up with theories and hypotheses that may or may not reflect reality. This is why scientists have to do experiments, to establish empirical evidence that they are right (or possibly more often, that they are not right).

[Retra]

There's also the fact that there's no known upper limit on the complexity/time needed to solve an arbitrary problem. Just because you can create an AGI doesn't mean nearly impossible problems suddenly become tractable.

Humans can already create humans that can outsmart them, and only every so often do they solve an interesting problem here or there.

[781]

It's pretty obvious that if you want to simulate a brain, a Von Neumann architecture is terrible. People will invent new architectures, just like they invented GPUs and TPUs.

And your second point is also flawed, the fact that Moore's Law stops working doesn't prevent one from having ever more computing power, by simply buying more chips. Google doesn't run on one massive computer, but on millions of small ones.

[ben_w]

> It's pretty obvious that if you want to simulate a brain, a Von Neumann architecture is terrible. People will invent new architectures, just like they invented GPUs and TPUs.

It really isn’t. Sure, the Von Neumann architecture doesn’t match our brains, but right now silicon is so much faster than biology that our fundamental problem is elsewhere — our best understanding of what it means to learn from experience doesn’t allow us to make machines which learn as effectively as we do from as little data as we do.

And that is the point — we can only (usefully) invent a new architecture like we did TPUs when we have a better idea. Sure, we probably will, but to what schedule? Biology is not obligated to make sense to us. Despite my general optimism about AI, I have to accept the possibility that perhaps the rules governing our own intelligence could (in principle) be as incomprehensible to us as they are to any other primate.

> And your second point is also flawed, the fact that Moore's Law stops working doesn't prevent one from having ever more computing power, by simply buying more chips. Google doesn't run on one massive computer, but on millions of small ones.

You seem to have missed my point here, too. I explicitly suggested what you used as a counter-argument — millions of small computers working together. To be precise, I suggested using 217.52 million A12 SoC units, running at 5e12 op/s, and a (guesstimated) 5 W TDP. This gave me an estimated 35,465 real-time human brains at a power cost of 36.8 kW per brain, which consumed roughly 32,200 US dollars of electricity per year, even when making the over-optimistic assumption the chips were the only power requirement (i.e. no network, no cooling).

This also gave me a hardware supply cost of about $2,500,000 per brain (assuming the cost of the cheapest iPad using an A12, because I lack any better idea for how to estimate the cost of all other components needed to keep the chip working).

If you hit the atomic limit, you get a x900 improvement (I think) on those costs. Which is great [1] if we know enough about how our minds work to replicate them… and my point is that we don’t.

[1] except for the social and economic implications when minds powered by sunlight are cheaper than literal slaves given nothing more than the minimum food to keep them alive, which I also hope we’ll deal with but is a totally independent question.