[skissane]

> Lying is intentional, algorithms and computers do not have intentions. People can lie, computers can only execute their programmed instructions. Much of AI discourse is extremely confusing and confused because people keep attributing needs and intentions to computers and algorithms.

How do you know whether something has “intentions”? How can you know that humans have them but computer programs (including LLMs) don’t or can’t?

If one is a materialist/physicalist, one has to say that human intentions (assuming one agrees they exist, contra eliminativism) have to be reducible to or emergent from physical processes in the brain. If intentions can be reducible to/emergent from physical processes in the brain, why can’t they also be reducible to/emergent from a computer program, which is also ultimately a physical process (calculations on a CPU/GPU/etc)?

What if one is a non-materialist/non-physicalist? I don’t think that makes the question any easier to answer. For example, a substance dualist will insist that intentionality is inherently immaterial, and hence requires an immaterial soul. And yet, if one believes that, one has to say those immaterial souls somehow get attached to material human brains - why couldn’t one then be attached to an LLM (or the physical hardware it executes on), hence giving it the same intentionality that humans have?

I think this is one of those questions where if someone thinks the answer is obvious, that’s a sign they likely know far less about the topic than they think they do.

[soist]

You're using circular logic. You are assuming all physical processes are computational and then concluding that the brain is a computer even though that's exactly what you assumed to begin with. I don't find this argument convincing because I don't think that everything in the universe is a computer or a computation. The computational assumption is a totalizing ontology and metaphysics which leaves no room for further progress other than the construction of larger data centers and faster computers.

[skissane]

> You're using circular logic. You are assuming all physical processes are computational and then concluding that the brain is a computer even though that's exactly what you assumed to begin with.

No, I never assumed “all physical processes are computational”. I never said that in my comment and nothing I said in my comment relies on such an assumption.

What I’m claiming is (1) we lack consensus on what “intentionality” is (2) we lack consensus on how we can determine whether something has it. Neither claim depends on any assumptions about “physical processes are computational”

If one assumes materialism/physicalism - and I personally don’t, but given most people do, I’ll assume it for the sake of the argument - intentionality must ultimately be physical. But I never said it must ultimately be computational. Computers are also (assuming physicalism) ultimately physical, so if both human brains and computers are ultimately physical, if the former have (ultimately physical) intentionality - why can’t the latter? That argument hinges on the idea both brains and computers are ultimately physical, not on any claim that the physical is computational.

Suppose, hypothetically, that intentionality while ultimately physical, involves some extra-special quantum mechanical process - as suggested by Penrose and Hameroff’s extremely controversial and speculative “orchestrated objective reduction” theory [0]. Well, in that case, a program/LLM running on a classical computer couldn’t have intentionality, but maybe one running on a quantum computer could, depending on exactly how this “extra-special quantum mechanical process” works. Maybe, a standard quantum computer would lack the “extra-special” part, but one could design a special kind of quantum computer that did have it.

But, my point is, we don’t actually know whether that theory is true or false. I think the majority of expert opinion in relevant disciplines doubts it is true, but nobody claims to be able to disprove it. In its current form, it is too vague to be disproven.

[0] https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...

[soist]

Intentions are not reducible to computational implementation because intentions are not algorithms that can be implemented with digital circuits. What can be implemented with computers and digital circuits are deterministic signal processors which always produce consistent outputs for indistinguishable inputs.

You seem to be saying that because we have no clear cut way of determining whether people have intentions then that means, by physical reductionism, algorithms could also have intentions. The limiting case of this kind of semantic hair splitting is that I can say this about anything. There is no way to determine if something is dead or alive, there is no definition that works in all cases and no test to determine whether something is truly dead or alive so it must be the case that algorithms might or might not be alive but because we can't tell then me might as well assume there will be a way to make algorithms that are alive.

It's possible to reach any nonsensical conclusion using your logic because I can always ask for a more stringent definition and a way to test whether some object or attribute satisfies all the requirements.

I don't know anything about theories of consciousness but that's another example of something which does not have an algorithmic implementation unless one uses circular logic and assumes that the brain is a computer and consciousness is just software.

[skissane]

> Intentions are not reducible to computational implementation because intentions are not algorithms that can be implemented with digital circuits.

What is an "intention"? Do we all agree on what it even is?

> What can be implemented with computers and digital circuits are deterministic signal processors which always produce consistent outputs for indistinguishable inputs.

We don't actually know whether humans are ultimately deterministic or not. It is exceedingly difficult, even impossible, to distinguish the apparent indeterminism of a sufficiently complex/chaotic deterministic system, from genuinely irreducible indeterminism. It is often assumed that classical systems have merely apparent indeterminism (pseudorandomness) whereas quantum systems have genuine indeterminism (true randomness), but we don't actually know that for sure – if many-worlds or hidden variables are true, then quantum indeterminism is ultimately deterministic too. Orchestrated objective reduction (OOR) assumes that QM is ultimately indeterministic, and there is some neuronal mechanism (microtubules are commonly suggested) which permits this quantum indeterminism to influence the operations of the brain.

However, if you provide your computer with a quantum noise input, then whether the results of computations relying on that noise input are deterministic depends on whether quantum randomness itself is deterministic. So, if OOR is correct in claiming that QM is ultimately indeterministic, and quantum indeterminism plays an important role in human intentionality, why couldn't an LLM sampled using a quantum random number generator also have that same intentionality?

> You seem to be saying that because we have no clear cut way of determining whether people have intentions then that means, by physical reductionism, algorithms could also have intentions.

Personally, I'm a subjective idealist, who believes that intentionality is an irreducible aspect of reality. So no, I don't believe in physical reductionism, nor do I believe that algorithms can have intentions by way of physical reductionism.

However, while I personally believe that subjective idealism is true, it is an extremely controversial philosophical position, which the clear majority of people reject (at least in the contemporary West) – so I can't claim "we know" it is true. Which is my whole point – we, collectively speaking, don't know much at all about intentionality, because we lack the consensus on what it is and what determines whether it is present.

> The limiting case of this kind of semantic hair splitting is that I can say this about anything. There is no way to determine if something is dead or alive, there is no definition that works in all cases and no test to determine whether something is truly dead or alive so it must be the case that algorithms might or might not be alive.

We have a reasonably clear consensus that animals and plants are alive, whereas ore deposits are not. (Although ore deposits, at least on Earth, may contain microscopic life–but the question is whether the ore deposit in itself is alive, as opposed being the home of lifeforms which are distinct from it.) However, there is genuine debate among biologists about whether viruses and prions should be classified as alive, not alive, or in some intermediate category. And more speculatively, there is also semantic debate about whether ecosystems are alive (as a kind of superorganism which is a living being beyond the mere sum of the individual life of each of its members) and also about whether artificial life is possible (and if so, how to determine whether any putative case of artificial life actually is alive or not). So, I think alive-vs-dead is actually rather similar to the question of intentionality – most people agree humans and at least some animals have intentionality, most people would agree that ore deposits don't, but other questions are much more disputed (e.g. could AIs have intentionality? do plants have intentionality?)

[soloist11]

> Personally, I'm a subjective idealist, who believes that intentionality is an irreducible aspect of reality. So no, I don't believe in physical reductionism, nor do I believe that algorithms can have intentions by way of physical reductionism.

I don't follow. If intentionality is an irreducible aspect of reality then algorithms as part of reality must also have it as realizable objects with their own irreducible aspects.

I don't think algorithms can have intentionality because algorithms are arithmetic operations implemented on digital computers and arithmetic operations, no matter how they are stacked, do not have intentions. It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers. As compositions of elementary operations there must be some element in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some magical quality of computers and I don't think computers have any magical qualities other than domains for digital circuits and numeric computation.

[drdeca]

> It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers.

I don't see how that makes it a category error? Like, assuming that numbers and arithmetic operations of numbers don't have intentions, and assuming that algorithms having intentions would imply that numbers and arithmetic operations have them, afaict, we would only get the conclusion "algorithms do not have intentions", not "attributing intentions to algorithms is a category error".

Suppose we replace "numbers" with "atoms" and "computers" with "chemicals" in what you said.

This yields "As compositions of [atoms] there must be some [element (in the sense of part, not necessarily in the sense of an element of the periodic table)] in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some magical quality of [chemicals] and I don't think [chemicals] have any magical qualities other than [...]." .

What about this substitution changes the validity of the argument? Is it because you do think that atoms or chemicals have "magical qualities" ? I don't think this is what you mean, or at least, you probably wouldn't call the properties in question "magical". (Though maybe you also disagree that people are comprised of atoms (That's not a jab. I would probably agree with that.)) So, let's try the original statement, but without "magical".

"As compositions of elementary operations there must be some element in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some [suitable-for-emergent-intentionality] quality of computers and I don't think computers have any [suitable-for-emergent-intentionality] qualities [(though they do have properties for allowing computations)]."

If you believe that humans are comprised of atoms, and that atoms lack intentionality, and that humans have intentionality, presumably you believe that atoms have [suitable-for-emergent-intentionality] qualities.

One thing I think is relevant here, is "we have nothing showing us that there exist [x]" and "it cannot be that there exists [x]" .

Even if we have nothing to demonstrate to us that numbers-and-operations-on-them have the suitable-for-emergent-intentionality qualities, that doesn't demonstrate that they don't.

That doesn't mean we should believe that they do. If you have strong priors that they don't, that seems fine. But I don't think you've really given much of a reason that others should be convinced that they don't?

[skissane]

From my own idealist viewpoint – all that ultimately exists is minds and the contents of minds (which includes all the experiences of minds), and patterns in mind-contents; and intentionality is a particular type of mind-content. Material/physical objects, processes, events and laws, are themselves just mind-content and patterns in mind-content. A materialist would say that the mind is emergent from or reducible to the brain. I would do a 180 on that arrow of emergence/reduction, and say that the brain, and indeed all physical matter and physical reality, is emergent from or reducible to minds.

If I hold a rock in my hand, that is emergent from or reducible to mind (my mind and its content, and the minds and mind-contents of everyone else who ever somehow experiences that rock); and all of my body, including my brain, is emergent from or reducible to mind. However, this emergence/reduction takes on a somewhat different character for different physical objects; and when it comes to the brain, it takes a rather special form – my brain is emergent from or reducible to my mind in a special way, such that a certain correspondence exists between external observations of my brain (both my own and those of other minds) and my own internal mental experiences, which doesn't exist for other physical objects. The brain, like every other physical object, is just a pattern in mind-contents, and this special correspondence is also just a pattern in mind-contents, even if a rather special pattern.

So, coming to AIs – can AIs have minds? My personal answer: having a certain character of relationship with other human beings gives me the conviction that I must be interacting with a mind like myself, instead of with a philosophical zombie – that solipsism must be false, at least with respect to that particular person. Hence, if anyone had that kind of a relationship with an AI, that AI must have a mind, and hence have genuine intentionality. The fact that the AI "is" a computer program is irrelevant; just as my brain is not my mind, rather my brain is a product of my mind, in the same way, the computer program would not be the mind of the AI, rather the computer program is a product of the AI's mind.

I don't think current generation AIs actually have real intentionality, as opposed to pseudo-intentionality – they sometimes act like they have intentionality, they lack the inner reality of it. But that's not because they are programs or algorithms, that is because they lack the character of relationship with any other mind that would require that mind to say that solipsism is false with respect to them. If current AIs lack that kind of relationship, that may be less about the nature of the technology (the LLM architecture/etc), and more about how they are trained (e.g. intentionally trained to act in inhuman ways, either out of "safety" concerns, or else because acting that way just wasn't an objective of their training).

(The lack of long-term memory in current generation LLMs is a rather severe limitation on their capacity to act in a manner which would make humans ascribe minds to them–but you can use function calling to augment the LLM with a read-write long-term memory, and suddenly that limitation no longer applies, at least not in principle.)

> I don't think algorithms can have intentionality because algorithms are arithmetic operations implemented on digital computers and arithmetic operations, no matter how they are stacked, do not have intentions. It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers

I disagree. To me, physical objects/events/processes are one type of pattern in mind-contents, and abstract entities such as numbers or algorithms are also patterns in mind-contents, just a different type of pattern. To me, the number 7 and the planet Venus are different species but still the same genus, whereas most would view them as completely different genera. (I'm using the word species and genus here in the traditional philosophical sense, not the modern biological sense, although the latter is historically descended from the former.)

And that's the thing – to me, intentionality cannot be reducible to or emergent from either brains or algorithms. Rather, brains and algorithms are reducible to or emergent from minds and their mind-contents (intentionality included), and the difference between a mindless program (which can at best have pseudo-intentionality) and an AI with a mind (which would have genuine intentionality) is that in the latter case there exists a mind having a special kind of relationship with a particular program, whereas in the former case no mind has that kind of relationship with that program (although many minds have other kinds of relationships with it)

I think everything I'm saying here makes sense (well at least it does to me) but I think for most people what I am saying is like someone speaking a foreign language – and a rather peculiar one which seems to use the same words as your native tongue, yet gives them very different and unfamiliar meanings. And what I'm saying is so extremely controversial, that whether or not I personally know it to be true, I can't possibly claim that we collectively know it to be true