[qsort]

> The definition of words changes in response to increasing knowledge

The usual process in mathematics and science is that you have a phenomenon that everyone agree exists but nobody can quite put their finger on it, so someone proposes a formal definition and if that definition turns out to be adequate, people work on the formal definition, and that's much easier because you now can use math, statistics, formal methods, etc; a prime example of this is the notion of "computability".

I don't believe that we are seeing the same thing with the concept of "intelligence", this is probably in part because it's much harder to capture the concept in a formal definition. Computers do computable stuff. Overlapping that notion with "intelligence" serves no purpose in my opinion: it explains nothing, it doesn't clarify anything, and it's certainly not obvious that the two are related.

> which are not "just" machine code either

I'm using "machine code" as proxy for "instructions/lambdas/whatever for a computational model of your choice", which they certainly are.

> The more you stress the simplicity of these models, the more intriguing their achievements seem.

It's not my intention to downplay any of the achievements of "AI". They are certainly not less intriguing when viewed from my perspective, the same way a compiler is not less intriguing if you think it's "just code".

My point is that any association of a formal concept (math, models, etc.) with philosophical concepts (intelligence, "truths about the world", consciousness, etc.) is always on thin ice, because natural language and formal concepts are hard to mix. Especially so when the concepts at play are so ephemeral.

[adamisom]

In the past, when a construct like 'intelligence' has been hard to pin down, science moves on--leaves it to 'philosophy' and works with formal definitions.

Would you say that's one part of your point? Just to clarify, I am only responding to that part of your point.

By way of example, behaviorists declared a strict subset of the human experience to be in the purview of scientific study--and that may even have been just fine for that era.

You seem to be sweeping something important under the rug because it's hard to pin down, and saying that this is what science has done in the past--and if so, you're right.

But there's an assumption--an assumption that it is safe to sweep things under the rug like that. That assumption may prove false, and if it does, we're screwed.

Convinced that AGI is not something to worry about? Fine--but surely you agree there's such a thing as an information hazard? That is, information that can be deadly in the wrong hands: like how to create the next COVID, or how to make a nuclear bomb. In past eras of human history, knowledge was not as powerful. Today and ever more so in the future, whether humanity can Get Things Right will matter.

So from my perspective, it doesn't matter that whatever 'intelligence' is, is hard to pin down: it's still got to be figured out, whether or not it's difficult.

[qsort]

> In the past, when a construct like 'intelligence' has been hard to pin down, science moves on--leaves it to 'philosophy' and works with formal definitions.

Yes, that's part of the point, your wording is better than mine. If we're sticking to a purely historical perspective, this is definitely what happened. Most disciplines that today we (rightfully) regard as fully independent, originally splintered off philosophy (the most obvious examples are mathematics and physics, but even something like economics, in spite of having become more formalized recently, undoubtedly originates from moral philosophy).

> You seem to be sweeping something important under the rug because it's hard to pin down, and saying that this is what science has done in the past--and if so, you're right.

I won't deny that strictly speaking there is a bit of inductivism at play here. Historically, the scientific approach of limiting the domain of discourse to a tractable subset has been so much more productive and successful than any alternative that my, as it were, "bayesian prior", is that we should replicate the same approach if possible at all.

> So from my perspective, it doesn't matter that whatever 'intelligence' is, is hard to pin down: it's still got to be figured out, whether or not it's difficult.

This is a reasonable position, but wouldn't you agree that it's more of a "moral intuition" (not that there's anything wrong with that!) than a position regarding how ML results ought to be interpreted? As such I have no real counterpoints to offer, except perhaps an utilitarian point of view: are you really sure that banging your head against this very specific wall is the most productive thing to do?

Most problems I see with AI arise from either flat out using the models incorrectly (i.e. mathematically wrong, not ethically wrong, which is what I was pointing out in my previous comments) or from already familiar "political" problems, i.e. incentives, transparency, privacy, openness of the decision-making process.

The good news is that none of that is new. The bad news is that our track record as a species on problems of that kind is abysmal. I doubt that, in any case, trying to halt scientific progress makes sense.

[Retric]

People learn to hit a target by changing the structure of their brain to fit the task. Computers become better at hitting a target by changing a data structure. That seems directly analogous to me. Critically, learning doesn’t imply the ability to perfectly execute the task.

[qsort]

I could not imagine two things that are more unrelated.

[Retric]

In what ways are they unrelated? Learning is just another activity that people do. Like sleeping or assembling ikea furniture.

[mannykannot]

Your response on definitions actually supports my point on the matter: definitions follow from knowledge ("a phenomenon that everyone agree exists") and are modified in response to new knowledge ("if that definition turns out to be adequate..." - and if not?) As before, "energy" stands as an example of how it works, and "computability" did not enter the lexicon until there was a use for it.

Nevertheless, I agree that in the specific case of current AI, using the word "intelligence" is misleading. I do not, however, think this misuse has any serious consequences, as, to reverse how I put it before, usage does not establish truths about the world.

>> which are not "just" machine code either

> I'm using "machine code" as proxy for "instructions/lambdas/whatever for a computational model of your choice", which they certainly are.

Then that is an unfortunate choice of proxy, unless, perhaps, you intended to imply that it is a priori impossible for intelligence to be created by running x86 machine code. It was not clear to me whether, by introducing machine code into the discussion, you were not making some sort of argument from incredulity against the possibility of AI.

> My point is that any association of a formal concept (math, models, etc.) with philosophical concepts (intelligence, "truths about the world", consciousness, etc.) is always on thin ice, because natural language and formal concepts are hard to mix. Especially so when the concepts at play are so ephemeral.

At least since Newton, mathematical models have proved very useful in discerning truths about the world. Are we to just assume they will not work for the biological phenomena of intelligence and consciousness?

[qsort]

> Your response on definitions actually supports my point on the matter

I'm afraid I failed to understand your point, then. I don't have a problem with what you said there.

> using the word "intelligence" is misleading. I do not, however, think this misuse has any serious consequences

This is where I fundamentally differ. Its misuse implies a connection between a formal model (algorithm expressed in a computational model) and a philosophical concept (intelligence) that's dubious at best. On a conceptual level, this makes it harder to reason clearly about those fundamentally mathematical and abstract concepts, and on a concrete level, it misleads the public at large, implying that certain goals have been reached when that's plainly untrue. That's pretty "serious" in my book.

> Then that is an unfortunate choice of proxy, unless, perhaps, you intended to imply that it is a priori impossible for intelligence to be created by running x86 machine code.

Again, I'm afraid I don't understand your objection. It's widely accepted that all reasonable computational models are equivalent. Citing x86 was colorful language, it has clearly no bearing on the point at large. Machine Learning algorithms are clearly computable, which means they are expressible as Turing machines, terms of a classical untyped lambda calculus, Python scripts, C++ template metaprograms, or anything else. They are literally just programs.

> At least since Newton, mathematical models have proved very useful in discerning "truths about the world." Are we to just assume they will not work for the biological phenomena of intelligence and consciousness?

I certainly believe mathematical models to be useful, you would be hard pressed to say otherwise. The ontological status of scientific theories is however at the very least a debatable topic. One needs not believe Newtonian mechanics is ontologically true, it's a tenable position to claim it's just a model, and we accept that model because it's useful.

Specifically, one could easily argue that Newtonian mechanics is false, because, for example, it fails to accurately predict Mercury's orbit.

Similarly, one needs not believe ML is anything more than relatively simple math to find it useful.

[mannykannot]

> I'm afraid I failed to understand your point, then. I don't have a problem with what you said there.

You have to go back a couple of posts to see the point. There, you wrote "'Intelligence' is a word that, etymologically and semantically, is related to human or human-like capabilities. You wouldn't say that a leaf floating on a lake is swimming." As we are now agreed that definitions follow from knowledge and are modified in response to new knowledge, it would not be somehow wrong to extend the concept of intelligence to a certain class of machines, if it turns out to be useful and informative to do so.

> They are literally just programs.

I take it, then, that you don't agree with the sort-of Platonist view that algorithms have an existence independently of any implementation? I'm on the fence, myself, but lean towards the Platonist side.

Regardless, it follows from your position here that your original statement "What's happening in my brain is something we don't have full scientific knowledge of, but we know it's not x86 machine code" can be rewritten as "What's happening in my brain is something we don't have full scientific knowledge of, but we know it's not computable." - but while the former is true, the status of the latter is not yet decided, so they are not identical propositions.

> One needs not believe Newtonian mechanics is ontologically true, it's a tenable position to claim it's just a model, and we accept that model because it's useful.

One could say the same about specific ontologies - they are as subject to revision in the face of increasing knowledge as are both mathematical models and individual words - and if it turns out that a mathematical model of biological intelligence or consciousness is effective and useful, it would be tendentious to imagine an ontological line between that model and intelligence.

[qsort]

> it would not be somehow wrong to extend the concept of intelligence to a certain class of machines, if it turns out to be useful and informative to do so.

No objections, but it's a pretty big "if". You could restate my point as "there is no evidence it is in fact useful".

> I take it, then, that you don't agree with the sort-of Platonist view that algorithms have an existence independently of any implementation? I'm on the fence, myself, but lean towards the Platonist side.

I don't, my position is essentially formalist. While I believe most research mathematicians would side with me here, your position is absolutely valid; famously, Kurt Godel was a Platonist, as are many others. My only observation here is that even from a Platonist point of view you are not really rejecting formalism, at least in the sense that while you don't agree it's the only view, I find it impossible to argue that one can't view mathematical objects as formal constructs.

> "What's happening in my brain is something we don't have full scientific knowledge of, but we know it's not computable." - but while the former is true, the status of the latter is not yet decided, so they are not identical propositions.

No, I don't really hold that view. I probably worded that badly. My position is that the latter is unknown, and I would be content to accept that my brain is not fundamentally different than an algorithm if you showed me an algorithm that can effectively emulate my brain within an acceptable margin of error. Ironically, if it were possible to do that, it would be proof there is no "intelligence", only "computability", making the first entirely redundant.

> One could say the same about specific ontologies -

Again, this is really thin ice. I don't really know what to think about this because it's getting too abstract for my monkey brain, but it's certainly not obvious that a better model implies an ontological line between itself and reality. To put it bluntly, is a better model really "more true", i.e. qualitatively different from a worse one?

I'll take the seventh.

[magusdei]

Can you specify what you mean by "emulate my brain within an acceptable margin of error"? What would this mean as an actual experiment? Depending on your answer, I think we can actually test your implicit proposition that no such algorithm exists.

[qsort]

I'm not actually claiming no such algorithm exists, as I have already stated, but simply that such a thing is not known to exist.

I'm not an expert in neurosciences so I can only give an informal description. Let's also remove "me" from the equation, let's talk about a randomly chosen human H. We know for a fact that there is nothing physically special about human brains, they are just ordinary organic matter. This matter forms a system subject to the laws of physics. With enough computational power and scientific knowledge (we have neither as of now, AFAIK), we could write a program for a quantum Turing machine that runs a 1:1 simulation of H's brain in software. Any quantum program can be emulated by a Turing machine equipped with sufficient random numbers with at most an exponential slowdown, making this program computable in exactly the classical sense.

My questions are (1) is it possible, even in principle, to make a program of this kind? (2) Would such a program be sufficiently predictive (with any statistical notion of that concept you prefer) of H's behavior?

If there exists a program that satisfies both (1) and (2), then I'm content with the notion that I am, myself, not significantly different from such a program.

[mannykannot]

> You could restate my point as "there is no evidence it is in fact useful"

And if you had originally stated your point that way, I would probably pointed out that there is equally no evidence that it will not be useful, if it turns out to be the case.

> ...but it's certainly not obvious that a better model implies an ontological line between itself and reality.

Clearly, I failed to get my point across, to the point where I cannot guess where this question is coming from. Let's see if I can be clearer...

My position on the definitions of words is that they are contingent on our knowledge and that new knowledge can change our definitions (I gave 'energy' as an example, and you appear to have accepted this point a couple of posts back.)

I take the same view of ontologies; the categories we see are contingent on what we know and may change as our knowledge increases. This should not be surprising, given that ontologies are specific cases of words with meanings that nominally pertain to how the world is. There is no implication here that a better model implies an ontological line between itself and reality; rather, the point here is effectively a "so what" reply to your statement, "it's a tenable position to claim [Newtonian mechanics] is just a model, and we accept that model because it's useful." Mutatis mutandis, as they say, and consequently, there is no justification for holding on to old ontologies if new facts suggest a better alternative, any more than there is for models or theories.

> To put it bluntly, is a better model really "more true", i.e. qualitatively different from a worse one?

Did you mean to write that, especially given that, in your previous post, you offered an argument for the proposition that "Newtonian mechanics is false, because, for example, it fails to accurately predict Mercury's orbit." If there is a relevant point here, I think it is that "more true" models are qualitatively better (and quantitatively better, also.)

> Ironically, if it were possible to [show an algorithm that can effectively emulate your brain within an acceptable margin of error], it would be proof there is no "intelligence", only "computability", making the first entirely redundant.

How so? If "intelligence" is a useful concept now (and your objection to "artificial intelligence" seems to be predicated on it being so), when we do not know if the mind is computationally modelable, why would this usefulness necessarily vanish if this turns out to be the case?

> I'll take the seventh.

? - I'm not familiar with this expression.

[qsort]

> And if you had originally stated your point that way, I would probably pointed out that there is equally no evidence that it will not be useful, if it turns out to be the case.

No objections, but isn't it a bit weird to argue that we should do that just in case it might be useful someday? We'll deal with it when it comes up.

> Clearly, I failed to get my point across

I'm sorry, I'm pretty sure there's an argument but I just don't get it. I'm not really following the train of thought anymore.

>> I'll take the seventh. >? - I'm not familiar with this expression.

Play on words: https://en.wikipedia.org/wiki/Fifth_Amendment_to_the_United_...

https://en.wikipedia.org/wiki/Tractatus_Logico-Philosophicus