[Jeff_Brown]

By "math" I mean proving theorems, not doing arithmetic. Yes, we're better at finding useful theorems, but computers can do it.

More generally, the fact that currently humans are the only entity observed doing X does not mean you need to understand humans to understand X.

[randomidiot666]

I wrote "computation", not "arithmetic". Human intelligence goes beyond computation / mathematical logic, and you seem to ignore all of that. We haven't got a clue how consciousness works. It's a total mystery.

[Jeff_Brown]

The aim of AI is intelligence, not human intelligence. It's not to emulate a process; it's to solve problems.

If we do build AI, maybe we'll never know if it's conscious. You can't know whether any other human is conscious, either. But you can know whether they make you laugh, or cry, or learn, or love. The knowable things are good enough.

[yters]

What if the secret sauce that makes intelligence, the kind that invents AI, is consciousness? I, at least, certainly do a lot of conscious thinking when I solve problems, as opposed to unconscious thinking :)

[perl4ever]

"I, at least, certainly do a lot of conscious thinking when I solve problems"

That jumps out at me, because I do a lot of "unconscious thinking" to solve problems and I feel like I've read where other people describe similar experiences.

Besides the cliche of solving problems in your sleep, I sometimes have an experience where consciously focusing on solving a problem leads to a blind alley, and distracting my conscious mind with something else somehow lets a background task run to "defrag" or something. But on the other hand there is "bad" distraction too - I'm not sure offhand what the difference is.

It's possible that I'm far from typical, but I also suspect people of different types and intellects might process things in very different ways too.

But to me, I definitely have a strong sense much of the time that my conscious mind engages in the receipt of information about something complex and then the actual analysis is happening somewhere invisible to me in my brain. I'm frequently conscious that I'm figuring something out and yet unaware of the process.

It particularly seems weird to me that other people often seem to be convinced they are conscious of their thought processes, because surely the type of person who is not a knowledge worker isn't? I'm not sure if my way of thinking is the "smart way", the "dumb way", or just weird, but I'm sure that there is significant diversity among people in general.

Sometimes I wonder if the model of AI is the typical mind of a very small subset of humanity that's unlike the rest, kind of like the way psychological experiments have been biased towards college students since that's who they could easily get.

[yters]

I've never solved a problem while completely unconscious. I've occasionally had insights while dreaming, and there is some intuitive aspect to thought that is difficult or impossible to explicitly articulate. But, every instance of problem solving I engage in is connected with conscious thought.

[perl4ever]

I don't know what "completely unconscious" means, but it doesn't sound like what I was describing.

I think I agree that my problem solving is connected with conscious thought, but the heavy lifting is mostly (or at least frequently) done by something that "I" am not aware of in detail.

When someone is explaining something complicated, pretty often, maybe not always, my (conscious) mind is pretty blank. I can say "yeah, I'm following you", but I feel like I'm not. Then when I start working on it, I feel like I am fumbling around for the keys to unlock some background processing that was happening in the meantime.

Also, when I am in a state where I am consciously writing something elaborate, and I feel connected to the complex concepts behind it, sometimes I get stuck in a blind alley. My context seems too narrow, and often I can get unstuck by just doing something unrelated to distract my conscious mind, like browsing news on my phone and then it's like a stuck process was terminated and I realize what I need to change on a higher level of abstraction.

It's possible I have some sort of inherent disability that I am compensating for by using a different part of my brain than normal, I suppose.

[randomidiot666]

But I wasn't talking about Artificial Intelligence or problem solving. I am talking about Actual Intelligence, specifically human level intelligence.

If we build AI we could only know if its conscious if we know what conscious is, and that is something we do not know, and perhaps will never know. It could be fundamentally beyond our comprehension.