| I think the conversation derailed a bit. I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. My original comment was that it seems (and it is actually documented in the books I referenced) that the AI research space builds its claims on assumptions, not on facts, and that those assumptions are flawed. So a nice discussion, to begin with, would consider: 1) why I make the claim that the AI research space builds its claims on assumptions instead of facts, why we could say that there are actually no assumptions but facts, or why the assumptions are correct. 2) instead of strictly and directly dismissing readings on philosophy, I would expect intelligent and curious people to embrace new references. Particularly if those references are highly regarded and a solid contribution during the last 120 years Regarding point 1), I can barely count a single comment in this thread that tries to engage in the idea of the assumptions (except for some comments that agree with the premise). Then regarding point 2), I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange. It looks like if the space revived during the 2000s with the invention of neural nets (RL, GAN, etc), and then became isolated from contributions about human intelligence, even though it continually tries to explain intelligence in its own terms. The reference to What Computers Can't Still do is precisely relevant because it narrates exactly this same discussion (false assumptions, claims built upon assumptions instead of facts, dismissal of evidence from psychology, dismissal of frameworks from philosophy, fallacies about progress), but it was written in 1972. Still, you read the book today, and it is totally relevant. Now, regarding your comments: > Do LLM experience objectively or not experience at all? How can you say? The world cannot be experienced 'objectively'. If they experience the world most probably you won't notice. Given that the only way of interacting with an LLM is through a process initiated solely by a human actor, it would be difficult to assess whether an LLM experiences anything at all. > I can say that they could think. The thought process implies a measurable product. Yes, there are similar situation with it, it can be hard to say sometimes if we observe a product of a thought or something else. But science has some success with this, like claiming that bees can think and solve problems. The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine. There is, as far as I know, no current evidence that human beings process information the same way a computer does it, nor that though processes necessarily imply a measurable outcome. > I'm telling you, psychologists (who specialize on mind research) do not know what consciousness is and they do not have a definition of a subjective experience (well, if we treat your proposed definition as a valid definition, then we should say they have plenty). And my claim still stands: until you heard about new science "Subjectology" you can be sure that no one knows what subjective experience is. Including linguists. Psychology is a very broad field with lights and shadows through its short history. Here, and in my original comment, I am not talking about linguists in general, but specifically about cognitive linguistics. The contributions made be the field are significant and mostly lacking in AI research (for example, the idea of embodiment, the rebuttal of generative grammars, prototype theory, frame semantics, among others). What you mention as 'subjectology', would be just psychology. Foucault explains more or less clearly why this cannot be a science, and that's just fine (in The Order of Things). > You shouldn't believe that anything can be settled in the philosophy space. Philosophers can think they have settled things, but until science agreed and started empirical studies, it is just philosophers believing that they settled things. Well... certainly nothing can be 'settled' (not even in science, btw), but my point is: there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us. Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation. |
I can probably to clarify it.
1. I don't want to end in a situation when AI deserves human rights but I deny it. There are moral reasons for that, and they are important.
2. I employ a systemic view. Not just arguments for consciousness or against it. I look at the people generating these arguments, how their minds work? I look at social institutions while they try to find some consensus. I'm very interested in their inner processes of generating truths. IN particular I'm interested in their failings and how they can generate untruths instead.
3. To understand a system I rely on historic data. How people and social institutions (including science) dealt with similar questions before.
The issue is, that the problem of LLM agency has potentially extremely wide implications, I expect people to be afraid of them, I expect social institutions to be afraid of them, so I cannot trust science in this regard like I trust it when it talks about physics or biochemistry.
> 2) instead of strictly and directly dismissing readings on philosophy
I think I have a good idea what philosophy thinks now, and it doesn't seem convincing. It looks like a normal philosophy, not like an established science, so you should take it with a grain of salt.
> The world cannot be experienced 'objectively'.
So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only possible way?
> I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange.
I believe it is an expected outcome. AI is evolving fast, there are plenty of things to research without establishing connections with other branches of science. No sane AI researcher would stop researching AI to get PhD in linguistics to build a bridge between AI research and linguistics. Probably in an ideal world this shouldn't happen, maybe it is short-sighted behavior of a system, but it is just how things work in our real world.
BTW it is a good example of what happens with all the philosophy when shit hits the fan. When possibility of empirical studies arrives, no one bothers themselves with philosophy of things.
> I am not talking about linguists in general, but specifically about cognitive linguistics.
I studied psychology, I've read some linguists (cognitive ones, because they are in an adjacent field), and you see, I don't have trust in either. They do their research, they find some interesting facts and devise interesting theories, but it is all looks more like a chemistry in the first half of a XIX century, than a chemistry after periodic table was created. They can't find their building blocks to create a sound theory.
> The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine.
No, I'm not implying "modeling a thinking process". We don't know what thinking process is. What we observe in our minds is not thinking by itself, it is some kind of a mirror process in our consciousness. The real thinking is hidden from us, but it creates echoes in our consciousness we can observe. If we don't know how thinking works, we can't model it. BTW the reverse is also true: if we can't model thinking, we don't know how it works.
I'm defining thinking more in terms of a problem solving ability. Like psychologists do. Science still doesn't have a good enough definition for thinking, but it has some definitions that a) operational; b) good enough for some limited tasks. "Operational" means that they are defined in terms how to measure what you define, not in terms of modeling some process.
> there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us.
Well, I don't argue that current LLM systems do not possess agency or experience, I argue that we should not trust philosophy to be the first who claims that LLM systems got agency, if they really got it. There is a possibility that they will fight to the death against it even if it is true. You see, until philosophy methods successfully proved that something is conscious despite it was deemed unconscious before, we can't really know that their methods really work. Maybe they work, or maybe they just mirror our biases and heuristics.
> Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation.
I hadn't read books you mentioned, but in your words I see nothing that can hint that those books have something I don't considered already. So maybe I'll read them in a future, but I wouldn't postpone my engagement in the conversation till I read them.