[soloist11]

> Personally, I'm a subjective idealist, who believes that intentionality is an irreducible aspect of reality. So no, I don't believe in physical reductionism, nor do I believe that algorithms can have intentions by way of physical reductionism.

I don't follow. If intentionality is an irreducible aspect of reality then algorithms as part of reality must also have it as realizable objects with their own irreducible aspects.

I don't think algorithms can have intentionality because algorithms are arithmetic operations implemented on digital computers and arithmetic operations, no matter how they are stacked, do not have intentions. It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers. As compositions of elementary operations there must be some element in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some magical quality of computers and I don't think computers have any magical qualities other than domains for digital circuits and numeric computation.

[drdeca]

> It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers.

I don't see how that makes it a category error? Like, assuming that numbers and arithmetic operations of numbers don't have intentions, and assuming that algorithms having intentions would imply that numbers and arithmetic operations have them, afaict, we would only get the conclusion "algorithms do not have intentions", not "attributing intentions to algorithms is a category error".

Suppose we replace "numbers" with "atoms" and "computers" with "chemicals" in what you said.

This yields "As compositions of [atoms] there must be some [element (in the sense of part, not necessarily in the sense of an element of the periodic table)] in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some magical quality of [chemicals] and I don't think [chemicals] have any magical qualities other than [...]." .

What about this substitution changes the validity of the argument? Is it because you do think that atoms or chemicals have "magical qualities" ? I don't think this is what you mean, or at least, you probably wouldn't call the properties in question "magical". (Though maybe you also disagree that people are comprised of atoms (That's not a jab. I would probably agree with that.)) So, let's try the original statement, but without "magical".

"As compositions of elementary operations there must be some element in the composite with intentionality or the claim is that it is an emergent property in which case it becomes another unfounded belief in some [suitable-for-emergent-intentionality] quality of computers and I don't think computers have any [suitable-for-emergent-intentionality] qualities [(though they do have properties for allowing computations)]."

If you believe that humans are comprised of atoms, and that atoms lack intentionality, and that humans have intentionality, presumably you believe that atoms have [suitable-for-emergent-intentionality] qualities.

One thing I think is relevant here, is "we have nothing showing us that there exist [x]" and "it cannot be that there exists [x]" .

Even if we have nothing to demonstrate to us that numbers-and-operations-on-them have the suitable-for-emergent-intentionality qualities, that doesn't demonstrate that they don't.

That doesn't mean we should believe that they do. If you have strong priors that they don't, that seems fine. But I don't think you've really given much of a reason that others should be convinced that they don't?

[soloist11]

I don't know what atoms and chemicals have to do with my argument but the substitutions you've made don't make sense and I would call it ill-typed. A composition of numbers is also a number but a composition of atoms is something else and not an atom so I didn't really follow the rest of your argument.

Computers have a formal theory and to say that a computer has intentions and can think would be equivalent to supplying a constructive proof (program) demonstrating conformance to a specification for thought and intention. These don't exist so from a constructive perspective it is valid to say that all claims of computers and software having intentions and thoughts are simply magical, confused, non-constructive, and ill-typed beliefs.

[skissane]

> A composition of numbers is also a number but a composition of atoms is something else and not an atom so I didn't really follow the rest of your argument.

That's not true. To give a trivial example, a set or sequence of numbers is composed of numbers but is not itself a number. 2 is a number, but {2,3,4} is not a number.

> Computers have a formal theory

They don't. Yes, there is a formal theory mathematicians and theoretical computer scientists have developed to model how computers work. However, that formal theory is strictly speaking false for real world computers – at best we can say it is approximately true for them.

Standard theoretical models of computation assume a closed system, determinism, and infinite time and space. Real world computers are an open system, are capable of indeterminism, and have strictly sub-infinite time and space. A theoretical computer and a real world computer are very different things – at best we can say that results from the former can sometimes be applied to the latter.

There are theoretical models of computation that incorporate nondeterminism. However, I'd question whether the specific type of nondeterminism found in such models, is actually the same type of nondeterminism that real world computers have or can have.

Even if you are right that a theoretical computer science computer can't have intentionality, you haven't demonstrated a real world computer can't have intentionality, because they are different things. You'd need to demonstrate that none of the real differences between the two could possibly grant one the intentionality the other lacks.

[soloist11]

> That's not true. To give a trivial example, a set or sequence of numbers is composed of numbers but is not itself a number. 2 is a number, but {2,3,4} is not a number.

That's still a number because everything in a digital computer is a number or an operation on a number. Sets are often encoded by binary bit strings and boolean operations on bitstrings then have a corresponding denotation as union, intersection, product, exponential, powerset, and so on.

[skissane]

> That's still a number because everything in a digital computer is a number or an operation on a number.

I feel like in this conversation you are equivocating over distinct but related concepts that happen to have the same name. For example, “numbers” in mathematics versus “numbers” in computers. They are different things - e.g. there are an infinite number of mathematical numbers but only a finite number of computer numbers - even considering bignums, there are only a finite number of bignums, since any bignum implementation only supports a finite physical address space.

In mathematics, a set of numbers is not itself number.

What about in digital computers? Well, digital computers don’t actually contain “numbers”, they contain electrical patterns which humans interpret as numbers. And it is a true that at that level of interpretation, we call those patterns “numbers”, because we see the correspondence between those patterns and mathematical numbers.

However, is it true that in a computer, a set of numbers is itself a number? Well, if I was storing a set of 8 bit numbers, I’d store them each in consecutive bytes, and I’d consider each to be a separate 8-bit number, not one big 8n-bit number. Of course, I could choose to view them as one big 8n-bit number - but conversely, any finite set of natural numbers can be viewed as a single natural number (by Gödel numbering); indeed, any finite set of computable or definable real numbers can be viewed as a single natural number (by similar constructions)-indeed, by such constructions even infinite sets of natural or real numbers can be equated to natural numbers, provided the set is computable/definable. However, “can be viewed as” is not the same thing as “is”. Furthermore, whether a sequence of n 8-bit numbers is n separate numbers or a single 8n-bit number is ultimately a subjective or conventional question rather than an objective one - the physical electrical signals are exactly the same in either case, it is just our choice as to how to interpret them

[soloist11]

> However, “can be viewed as” is not the same thing as “is”

Ultimate reality is fundamentally unknowable but what I said about computers and digital circuits is correct. We have a formal theory of computers and that is why we can construct them in factories. There is no such theory for people or the biosphere which is why when someone argues for intentionality or some other attribute possessed by both people and computers I discount whatever they are saying unless they can formally specify how some formal statement in a logical syntax (program) corresponds to the same attribute in people and animals.

This confusion between formal theories and informal concepts like intentionality is why I am generally wary of anyone who claims computers can think and possess intelligence. The ultimate endpoint of this line of reasoning is complete annihilation of the biosphere and its replacement with factories producing nothing but computers and power plants for shuttling electrons. The people who believe computers are a net positive might not think this way but by equating computers with people they are ultimately devaluing the irreducible complexity of what it means to be a living animal (person) in an ecology with irreducible properties and attributes.

I'm obviously not going to convince anyone who believes computers and algorithms can think and possess intelligence but it is clear to me that by elevating digital computers above biology and ecology they are devaluing their own humanity and justifying actions which will ultimately end in disaster.

[skissane]

From my own idealist viewpoint – all that ultimately exists is minds and the contents of minds (which includes all the experiences of minds), and patterns in mind-contents; and intentionality is a particular type of mind-content. Material/physical objects, processes, events and laws, are themselves just mind-content and patterns in mind-content. A materialist would say that the mind is emergent from or reducible to the brain. I would do a 180 on that arrow of emergence/reduction, and say that the brain, and indeed all physical matter and physical reality, is emergent from or reducible to minds.

If I hold a rock in my hand, that is emergent from or reducible to mind (my mind and its content, and the minds and mind-contents of everyone else who ever somehow experiences that rock); and all of my body, including my brain, is emergent from or reducible to mind. However, this emergence/reduction takes on a somewhat different character for different physical objects; and when it comes to the brain, it takes a rather special form – my brain is emergent from or reducible to my mind in a special way, such that a certain correspondence exists between external observations of my brain (both my own and those of other minds) and my own internal mental experiences, which doesn't exist for other physical objects. The brain, like every other physical object, is just a pattern in mind-contents, and this special correspondence is also just a pattern in mind-contents, even if a rather special pattern.

So, coming to AIs – can AIs have minds? My personal answer: having a certain character of relationship with other human beings gives me the conviction that I must be interacting with a mind like myself, instead of with a philosophical zombie – that solipsism must be false, at least with respect to that particular person. Hence, if anyone had that kind of a relationship with an AI, that AI must have a mind, and hence have genuine intentionality. The fact that the AI "is" a computer program is irrelevant; just as my brain is not my mind, rather my brain is a product of my mind, in the same way, the computer program would not be the mind of the AI, rather the computer program is a product of the AI's mind.

I don't think current generation AIs actually have real intentionality, as opposed to pseudo-intentionality – they sometimes act like they have intentionality, they lack the inner reality of it. But that's not because they are programs or algorithms, that is because they lack the character of relationship with any other mind that would require that mind to say that solipsism is false with respect to them. If current AIs lack that kind of relationship, that may be less about the nature of the technology (the LLM architecture/etc), and more about how they are trained (e.g. intentionally trained to act in inhuman ways, either out of "safety" concerns, or else because acting that way just wasn't an objective of their training).

(The lack of long-term memory in current generation LLMs is a rather severe limitation on their capacity to act in a manner which would make humans ascribe minds to them–but you can use function calling to augment the LLM with a read-write long-term memory, and suddenly that limitation no longer applies, at least not in principle.)

> I don't think algorithms can have intentionality because algorithms are arithmetic operations implemented on digital computers and arithmetic operations, no matter how they are stacked, do not have intentions. It's a category error to attribute intentions to algorithms because if an algorithm has intentions then so must numbers and arithmetic operations of numbers

I disagree. To me, physical objects/events/processes are one type of pattern in mind-contents, and abstract entities such as numbers or algorithms are also patterns in mind-contents, just a different type of pattern. To me, the number 7 and the planet Venus are different species but still the same genus, whereas most would view them as completely different genera. (I'm using the word species and genus here in the traditional philosophical sense, not the modern biological sense, although the latter is historically descended from the former.)

And that's the thing – to me, intentionality cannot be reducible to or emergent from either brains or algorithms. Rather, brains and algorithms are reducible to or emergent from minds and their mind-contents (intentionality included), and the difference between a mindless program (which can at best have pseudo-intentionality) and an AI with a mind (which would have genuine intentionality) is that in the latter case there exists a mind having a special kind of relationship with a particular program, whereas in the former case no mind has that kind of relationship with that program (although many minds have other kinds of relationships with it)

I think everything I'm saying here makes sense (well at least it does to me) but I think for most people what I am saying is like someone speaking a foreign language – and a rather peculiar one which seems to use the same words as your native tongue, yet gives them very different and unfamiliar meanings. And what I'm saying is so extremely controversial, that whether or not I personally know it to be true, I can't possibly claim that we collectively know it to be true

[soloist11]

My point is that when people say computers and software can have intentions they're stating an unfounded and often confused belief about what computers are capable of as domains for arithmetic operations. Furthermore, the Curry-Howard correspondence establishes an equivalence between proofs in formal systems and computer programs. So I don't consider what the social media gurus are saying about algorithms and AI to be truthful/verifiable/valid because to argue that computers can think and have intentions is equivalent to providing a proof/program which shows that thinking and intentionality can be expressed as a statement in some formal/symbolic/logical system and then implemented on a digital computer.

None of the people who claimed that LLMs were a hop and skip away from achieving human level intelligence ever made any formal statements in a logically verifiable syntax. They simply handwaved and made vague gestures about emergence which were essentially magical beliefs about computers and software.

What you have outlined about minds and patterns seems like what Leibniz and Spinoza wrote about but I don't really know much about their writing so I don't really think what you're saying is controversial. Many people would agree that there must be irreducible properties of reality that human minds are not capable of understanding in full generality.

[skissane]

> My point is that when people say computers and software can have intentions they're stating an unfounded and often confused belief about what computers are capable of as domains for arithmetic operations. Furthermore, the Curry-Howard correspondence establishes an equivalence between proofs in formal systems and computer programs

I'd question whether that correspondence applies to actual computers though, since actual computers aren't deterministic – random number generators are a thing, including non-pseudorandom ones. As I mentioned, we can even hook a computer up to a quantum source of randomness, although few bother, since there is little practical benefit, although if you hold certain beliefs about QM, you'd say it would make the computer's indeterminism more genuine and less merely apparent

Furthermore, real world computer programs – even when they don't use any non-pseudorandom source of randomness, very often interact with external reality (humans and the physical environment), which are themselves non-deterministic (at least apparently so, whether or not ultimately so) – in a continuous feedback loop of mutual influence.

Mathematical principles such as the Curry-Howard correspondence are only true with respect to actual real-world programs if we consider them under certain limiting assumptions–assume deterministic processing of well-defined pre-arranged input, e.g. a compiler processing a given file of source code. Their validity for the many real-world programs which violate those limiting assumptions is much more questionable.

[soloist11]

Even with a source of randomness the software for a computer has a formal syntax and this formal syntax must correspond to a logical formalism. Even if you include syntax for randomness it still corresponds to a proof because there are categorical semantics for stochastic systems, e.g. https://www.epatters.org/wiki/stats-ml/categorical-probabili....

[skissane]

> Even with a source of randomness the software for a computer has a formal syntax and this formal syntax must correspond to a logical formalism.

Real world computer software doesn't have a formal syntax.

Formal syntax is a model which exists in human minds, and is used by humans to model certain aspects of reality.

Real world computer software is a bunch of electrical signals (or stored charges or magnetic domains or whatever) in an electronic system.

The electrical signals/charges/etc don't have a "formal syntax". Rather, formal syntax is a tool human minds use to analyse them.

By the same argument, atoms have a "formal syntax", since we analyse them with theories of physics (the Standard Model/etc), which is expressed in mathematical notation, for which a formal syntax can be provided.

If your argument succeeds in proving that computer programs can't have intentionality, an essentially similar line of argument can be used to prove that human brains can't have intentionality either.

[soloist11]

> If your argument succeeds in proving that computer programs can't have intentionality, an essentially similar line of argument can be used to prove that human brains can't have intentionality either.

I don't see why that's true. There is no formal theory for biology, the complexity exceeds our capacity for modeling it with formal language but that's not true for computers. The formal theory of computation is why it is possible to have a sequence of operations for making the parts of a computer. It wouldn't be possible to build computers if that was not the case because there would be no way to build a chip fabrication plant without a formal theory. This is not the case for brains and biology in general. There is an irreducible complexity to life and the biosphere.