[adestefan]

Weizenbaum even wrote an entire book, Computer Power and Human Reason: From Judgment To Calculation, on how AI is a crank term for computer science. The basic premise is that humans are not machine so stop using language that treats them as such. Especially that computers can decide what comes next, but only a human can choose what to do.

The book also has one of the best and most succinct descriptions of Turing machines and the theoretical underpinning of computer science that I have ever read. Even if you’re an AI maximalist you should read the third chapter of the book.

[erikerikson]

> humans are not machine

Aren't we? Casual chains upon our matter produce emergent behaviors using the same physics and chemistry that our mechanistic creations rely upon.

Certainly those behaviors and results do not produce the same repeatable, predictable results as our clockworks but that is the whole point of the field of AI (as opposed to the marketing corruption/term that is currently in vogue, so GAI if you prefer), to produce system and algorithm structures designed with architecture and patterns more like our own.

Perhaps you believe in the ghost in the machine hypothesis? The magical soul that is more than the emergent evolving pattern produced across time by DNA replicators? That this undefinable, unmeasurable spirit makes us forever different?

[aftoprokrustes]

This is an hypothesis. Of course, it is obvious that a lot of what human/animals are and do can be explained or described by physical processes. But there is the "hard problem of consciousness", to which no one has a satisfying answer.

I feel sarcasm in your last paragraph, but it feels rather dishonest, in the sense that you voluntarily use rather crude and ridiculous formulations, as if it was the only alternative. But there is some arrogance in thinking that we, smart as we are, finally got the final answer about one of the hardest questions in philosophy and metaphysics. Materialism is not proven: it is a basic methodological assumption of modern science - meaning that we do not have the tools to either prove or disprove it. Newton, Gödel and lots of other renowed scientists are knowed to have opposed materialism. Let's accept that the question is open.

On this topic, I always recommend reading the report of the Galileo Commission, which is a manifesto by a wide range of scientists and philosophers to reduce the stigma associated with even questioning this fundamental dogma.

[fasterik]

Just because there are two hypotheses doesn't mean that we should assign them equal probability. Why should we believe that we will need to modify the laws of physics to explain consciousness when they already do a remarkably good job of predicting and explaining everything else we know about chemistry and biology? If someone has an alternative theory of consciousness that goes beyond the standard model of particle physics, the burden of proof is on them to produce the equations that explain how this new substance or force interacts with the known fields of the standard model. Of course we should never put 0 probability on anything, but I know where I would put my money given the empirical track record of our current theories.

[aftoprokrustes]

I actually think it is a bit more subtle than that. I am here really focusing on consciousness as a subjective experience, which seems to be of an entirely different nature than physical processes. We have a lot of examples of emergent properties of complex systems, but as far as I can tell all of them stay in the same "realm": the emergent properties of an ant colony, for instance, are physical, resulting from physical interactions.

With the subjective experience as an emergent property, things such as a society, a country, an economy could become conscious, in the sense of having a _subjective_ experience of their own existence as entities separate from the rest of the world. If we accept the "consciousness as an emergent property", we _have_ to accept that possibility. Which, to me, is not less wild or unlikely than, say, the "theory" of a field of consciousness "received" or "captured" by physical systems with certain properties, the same way a radio can receive radio programs. There are additional reasons to want to consider alternative explanations, but going into them would rrquire much more space - if interested, I would point to the report of the Galileo Commission.

So it does not change anything to physics, really: materialism is pretty much the best methodology to unpack the laws of physics: whatever you observe, see if you can find more elementary physical processes that explain it.

I am just a bit irritated by bold statements which assume that we know for certain that consciousness is an emergent property of physical processes. We do not, and the reason why this is such an accepted fact is more sociological than scientific - Newton and others decided to focus solely on physical processes as a methodological tool, and over centuries, the undeniable success of the approach in making discoveries _and_ building practical tools gave it an ontological status it did not have initially (Newton was for instance a very convinced Christian). Which makes me keen to remind that, because in the current scientific culture it is shameful to even ask the question.

[fasterik]

I'm not inside the scientific community so I can't verify the degree to which it's a shameful question. My guess is that it has a lot more to do with simple heuristics than dogmatism. A researcher has finite time and resources. They need to decide what to work on based on the likelihood that they can make progress on certain well-defined problems. We already have a centuries-long track record of making progress by studying things in terms of physical and chemical processes. That doesn't mean this approach can solve every problem, but there's not much else we can do until a new Einstein comes along and proposes an alternative that's compelling enough. I believe there are a lot of young scientists who would be willing to jump on a new paradigm if it was obviously leading to novel insights and breakthroughs, but that hasn't happened. It's not because these ideas are being suppressed, it's because nobody has put them forward in a rigorous and convincing manner.

[Vegenoid]

I can't say I find the Galileo Commission to be particularly compelling. It is authored by people who are inclined toward non-materialism, and I get the sense that they care more about consciousness existing beyond the physical brain than discovering what is true. This isn't to wholly dismiss it, there is good work and smart people behind it, but I think it has a pretty heavy agenda.

And, this may be petty, but the self-comparison to the maligned Galileo is rather off-putting.

[erikerikson]

Incidentally, you may enjoy Promethea[0] and its thesis that ideas are more real than any physical thing.

[0] https://en.wikipedia.org/wiki/Promethea

[erikerikson]

You've written your response thoughtfully, thank you.

Of course my description is of a hypothesis rather than ground truth.

It turns out that there isn't consensus over whether "The Hard Problem Of Consciousness" is actually a problem[0] but I need to admit I didn't know that before reading your comment. You may have been too kind in your assumption of my competence level on the subject. As adjunct, my writing was about my own currently highest probability expectations rather than a mature reporting of the state of the field of experts. It did not seem worth (then or now) writing to such a standard on a public discussion forum, even this one.

My last paragraph wasn't written with sarcasm (sorry to have given easy opportunity to read that in it) but it was a bit rushed, clumsily written, and unimaginative. Poorly written as it was, my curiosity was sincere (its been some time since I engaged with the state of the art) as was some of my dismissal. I have found it common that people have a belief in the uniqueness of humans as a platform for intelligence that frequently has its roots in sentiment and/or unexamined beliefs. Of course there are individuals whose thoughts meet a much higher bar. It is also the case that faith as a basis of belief does not invalidate the belief.

There is so much we don't know. We only recently explained how aspirin works[1]. That didn't stop it from working well during our long period of usage prior to our understanding. I'm comfortable with that an its analog here, that I don't understand consciousness and its mechanisms completely. I seem to experience it and it seems to result from what is, despite my incomplete knowledge of what that "what is" comprises. However, there is an imbalance of evidence for the material hypothesis and it seems plausible that emergent dynamics are sufficient to explain.

So... Yes, materialism is not proven and yet I currently hold it as the explanation that makes at least a partial contribution to the more complete truth. Further, that it is the explanation with the greatest volume of evidence and support. I suspect that it is sufficient for the emergence of intelligence and even consciousness in ourselves and as such sufficient for (through the same mechanisms) the emergence of intelligence and consciousness in our artificial constructs. Note that I also suspect we are still some distance from that inflection point.

Thank you for the reference to the Galileo Commission. I had not heard of it and am always happy to consider new perspectives and challenge those which I have held.

[0] https://survey2020.philpeople.org/ [1] https://pubmed.ncbi.nlm.nih.gov/14592543/

[stavros]

> Especially that computers can decide what comes next, but only a human can choose what to do.

I don't understand this, all the programs I've ever written make decisions based on some factors.

Are you talking about free will? If so, what is free will?

[johnnyworker]

snippet from the WP article on the book:

> Weizenbaum makes the crucial distinction between deciding and choosing. Deciding is a computational activity, something that can ultimately be programmed. It is the capacity to choose that ultimately makes one a human being. Choice, however, is the product of judgment, not calculation. Comprehensive human judgment is able to include non-mathematical factors such as emotions. Judgment can compare apples and oranges, and can do so without quantifying each fruit type and then reductively quantifying each to factors necessary for mathematical comparison.

Okay, so what is judgement? I haven't read that particular book and I don't quite remember his argument from interviews and lectures I saw, so this might be wrong, but I'd say it's for example saying "this is fair" when you measure the slices of pie you cut a cake into. That is, calculating that they're of equal size is pure computation; but there is no way to compute that when sharing cake with your friends, the slices should be equal.

Just like you can compute how much clean drinking water an average or specific person needs a day, with at least some accuracy, but when it comes to the question "should there be life in the universe" or "should people die of thirst", no computation could answer it. You could choose to write a program that decides it based on a random seed or a super complex algorithm taking a billion factors into account, but and then that program would decide the question, but it's essentially still something you did / chose.

[vidarh]

It's basically a religious view. For a "judgement" to be non-computable, it'd need to come from some factor in the human brain which violates know physics and can't be reproduced outside a human brain.

It's little more than arguing for a "soul" with no evidence for any effect that can't be explained by cause and effect.

[johnnyworker]

> For a "judgement" to be non-computable, it'd need to come from some factor in the human brain which violates know physics and can't be reproduced outside a human brain.

You say this as if we are even close to understanding much less reproducing the human brain completely, which probably would have to include the web of relations with all sorts of other living things that also go into the judgements we make, and the emotions we have while making them. Until you actually do draw the rest of the owl, it's not exactly "religious" to say there's no owl.

[vidarh]

No, it's an argument from logic that applies to any claim that any given entity can do things that are not computable.

> Until you actually do draw the rest of the owl, it's not exactly "religious" to say there's no owl.

The "real owl" here is to assume the human brain does something non-computable, in violation of all known physics and logic.

[johnnyworker]

You cannot compute what you don't understand, and even if you did by accident, you wouldn't know you computed it, as long as you don't understand what you're trying to do. That seems obvious to me.

And "computable" and "computable for us" are very different things. It's not about the machines or algorithms we might make one day, provided that we fully understand everything that goes into our our thoughts and emotions with nothing left unaccounted for, and everything turning out to be countable; it's about the ones we are actually making, back then and today, and then in some cases outsource our decisions to.

[aatd86]

Doing things because you can and not because you have to? Creative endeavors in the largest sense?

[arrrg]

Am I ever doing things because I can and not because I have to? Also, what mechanism determines what things I want to do because I can do them? And isn’t that mechanism then just not just another part of the machine.

Just because it feels as though I do things because I can doesn’t mean that is actually true.

[aatd86]

As long as you can imagine different possible futures and decide upon which one you want to try and realize, I think you have choice.

Choice stems from uncertainty, partial knowledge. It might be an illusion for an observer outside of the system, but as far as a participant within the system is concerned, there is choice, then there is free will.

I am writing this because I ca n but I don't need to do it. I have futures where I don't do that and do something more rewarding instead and still. As long as I am aware of the choices, then I have free will.

[vidarh]

This is the compatibilist view. But if it is an illusion, then that means the "choice" is computable and a computer can create the same outcomes.

[stavros]

Does a program have to do things? What can it do? What does a human have to/can do?

[aatd86]

Traditionally, a program is a series of instructions. The program doesn't really act on its own.

Now, a program which is objective driven and can infer from new inputs might be something else.

Just like humans try to maximize the stability of their structures via a reward system. (it got slighty complex, faulty at times, or the tradeoff between work vs reward is not always in favor of work because we do not control every variable, hence procrastination for example, or addiction which is not a conscious process but neuro-chemically induced).

[stavros]

But what does "act on its own" mean? If I give the program some randomness over its next action, is that "acting on its own"? When I'm at work, I act according to a series of instructions. Am I not acting on my own?

This is a very philosophical discussion, but if I had an infinitely-powerful computer and could simulate an entire universe based on a series of instructions (physical laws), would the beings in that universe that created societies not be "acting on their own"?

[aatd86]

Yes, as long as the computer chooses its next set of instructions in order to maximize a given value (the objective), I would say that it acts on its own. Instruction set that was never defined by anyone that is.

If the instruction set is limited and defined by someone else, I believe it doesn't.

I think, re. the simulated universe, that for us, they wouldn't have free will because we know causality (as long as we are all knowing about the simulation). But as far as they would be concerned, wouldn't they have free will if they know that they don't know everything and whether the future they imagine is realizable?

If they knew with certainty that something was not realizable, they wouldn't bother, but since they don't know, either they try to realize a given future or they don't.

Partial information provides choice of action, therefore free will.

[vidarh]

How do you imagine that act of choosing happens in your brain in a way that isn't computable?

[aatd86]

Computable by who? Because you don't have the full list of correlations and there are superlinear things (tail events) you'll get a probabilistic estimation at best.

[moffkalast]

Just because we have quantum RNG in our heads that doesn't make us automatically better. If anything it makes us worse since we don't act on reason alone.

[aatd86]

I don't know if there is a quantum rng or just an inference machine that manages to recognize patterns within input data and can do some math sometimes.

[adestefan]

A girl would like to ask a boy to the high school dance.

A computer can do all the calculations to decide on if it's a good idea. Given the inputs of the time they have spent together, the number of glances that are passed between then in the halls between classes, if he doesn't have a date yet or not, etc. The probability adds up to ask.

So the machine decides to ask.

The girl feels it. Has all the time they've spent together has made her feel a certain way? Maybe a weird tingle each time their arms touch. Is that glance in the hall this morning not just an accident, but him going a little out of his way for her to notice? She's asked around and knows that no one else has asked him, but doe he really not have a date yet? Can she overcome the bit of anxiety about asking a boy to the dance? Will she be able to accept the risk of rejection knowing that the chances may be high he says yes?

Only she can choose.

[holoduke]

All the tingles, feelings, anxieties and hesitations are activities triggered by little programs that work autonomously and are fully deterministic. The girl is fooled

[cmrdporcupine]

Even if you accept a strong determinist position, there is still a distinction:

The determining factors driving a computer program can be fully quanitified; the sets of inputs and conditions is finite, can be reasoned over, and described fully.

That's basically the fundamental description of computing, in fact, and what makes a Turing machine.

The determining factors "IRL" are effectively infinite, a causal "chain" of infinite (or near infinite) complexity that expands backwards to the Big Bang, (or whatever) and sideways around the planet and beyond. There is no catalog you could make of all the "causes" that could isolate things enough to truly reason over and describe them all.

And so, yeah, to say it's all just "little programs" is the most ridiculous reductionism, that basically purposefully neglects to see the complexity and depth of the world around us.

I personally take a strongly determinist, materialist philosophical position. But I would never ever express that in terms of "programs" or anything similar.

[d0mine]

You assume classic, non-quantum world.

The entropy of the observable universe is _finite_. https://physics.stackexchange.com/questions/4118/how-many-by...

[adestefan]

HN consistently reminds me of the park bench scene in Good Will Hunting.

[belter]

An illusory choice...

"Does Quantum Mechanics Rule Out Free Will?" - https://www.scientificamerican.com/article/does-quantum-mech...

[extractionmech]

“Me no likee” - Joseph Agassi

This post of yours sent me looking for this and I ended up with (now reading) Agassi’s provocative (& brutal) takedown of this book. As to the content of the book, Agassi pointedly mentioned Weiner. I will of course read Weizenbaum after this. (thanks.)

https://www.researchgate.net/publication/286058724_Computer_...

p.s. continuing on, the review turns very positive. Initial vitriol apparently more related to the “two worlds” matter.

[1vuio0pswjnm7]

Previous: https://news.ycombinator.com/item?id=36901158 [PDF]