| We may be talking past each other here and I'm not quite sure how to formulate my thoughts more clearly, but I'll try. >surely there's plenty of work it will do that fits within the bounds of the physics we know Simply to say some work would itself be a huge assumption, but plenty? That assumes far too much about our current state of knowledge, and it places artificial limitations on the capacity of a super-intelligence. The flaw in your thinking is your very reliance on our own knowledge as your frame of reference. You're extrapolating from that because you can't imagine anything else. But, that's the point: we don't know what we don't know (or what a super-intelligence might know). EDIT: I could attempt to use hypothetical scenarios here to convey how your assumptions might be wrong (e.g. what if this intelligence could jump dimensions such that our laws of physics no longer hold?). However, to do that would be to commit the same error that you're committing: it extrapolates back from our own limited language and understanding. By definition, if we can explain it, then it doesn't meet the test of the type of intelligence that I'm referencing./EDIT Your statement is also a pretty big tell that you're working from an assumption of only marginally increased intelligence, which is where I think our disconnect really enters. To appreciate the degree of intelligence I'm referencing, see [1]. It's long, but a good read in its own right. Specifically regarding this discussion, towards the middle of the page there are two graphs, labeled "Our Distorted View of Intelligence" and "Reality" which might help illustrate the scale of what I'm trying to communicate. Here also is an attendant quote: >In our world, smart means a 130 IQ and stupid means an 85 IQ—we don’t have a word for an IQ of 12,952 See what I mean? When intelligence reaches such a scale beyond our own, it destroys key assumptions. The idea that we can comprehend that the intelligence is doing something is not guaranteed. Further, even if perceived, that perception itself is potentially only incrementally more representative of true understanding than not perceiving it at all. So, for any meaningful purpose, our understanding would be much more closely aligned to that of the ants WRT the 10-lane superhighway. Another way to look at this is to imagine an infinitely expanding intelligence scale. At some zoom-level, we are pushed so much closer to the ants that the difference is virtually indiscernible. [1] http://waitbutwhy.com/2015/01/artificial-intelligence-revolu... |