[LudwigNagasena]

> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things

I don't see how any of the works you referenced can account for that either? Since when is the problem of consciousness solved and we can definitely say what does or doesn't result in consciousness?

[f_klem]

None of the works I mentioned solved or try to solve the mind/body problem or the issue of consciousness.

They are a frame of reference for not stepping into the common fallacies that the AI research field is based on.

[LudwigNagasena]

What's exactly the fallacy? How do the works help avoid stepping into that "fallacy" if they don't try to solve the issue of consciousness.

[f_klem]

The issue of consciousness appears when you think of the world in a mechanistic way: since all there is are laws of physics and materiality, then how could we explain our though processes and our perceptual experience? If the world itself (in a general, existential way) is only made of laws of physics and matter, the consciousness needs to be an emergent characteristic of physical systems, and needs to follow the laws of physics. Now at this stage, you are already in trouble and you need to explain what consciousness is and how it manifestates. And that's the moment where things like the computational theory of mind appears.

But you need to step back in order to detect the fallacy, one of which is: the brain/mind processes information like a computer, then we could build better computers that can think. This fallacy is assumed in the question 'can a machine think?'. There's another fallacy, which the author call the first step fallacy, which is common nowadays: we solved the language problem, then machines will be able to think in the near future.

So it is not about solving the consciousness problem, it is about not claiming things based on assumptions that can be easily challenged.

[LudwigNagasena]

Of course any research programme requires some assumptions. But I don’t see any reason to call it a fallacy. Saying that something may be “challenged” or is problematic is just weasel wording.

Either there are some serious issues that makes such theories ”flawed in the sense that they cannot account for subjective experience and agency, amongst other things”, or they are just normal theories.

[f_klem]

True, any research programme requires assumptions. The problem lies when those assumptions are either false or theoretical (unproved), and the community derives facts or claims from them.

Behind the actual AI programme operate the following assumptions (at least):

1. A biological assumption, that states that the brain works similar to a digital computer. The reality is that we do not know.

2. An epistemological assumption, that states that we know how our brain works (or an even worse assumption, that states that we don't even need to know how it works, it is sufficient to replicate its observed behavior). This is rather simplified, the assumption in reality being (as stated by Dreyfus) that we think all intelligent behavior can be formalized as heuristic rules (Dreyfus' critique is based on GOFAI, since the book is pre-GAN/RL AI systems). But the assumption still applies: we think all intelligent behavior can be sampled, captured and formalized in (albeit complex) statistical systems.

Dreyfus describes 4 or 5 in total, one of them is the psychological assumption, which states that the mind itself can be described as a digital computer (I think it might be outdated, since the actual debate is if something we could call 'mind' exists at all).

There is also a fallacy called first-step fallacy, which states that if the first step towards intelligence is met, then the rest of the steps are of similar nature (technical).

[LudwigNagasena]

You say that "the community" derives facts or claims from unproved assumptions, yet at the same time you say that you "strongly tend to disagree" with those theories and that the theories are "flawed in the sense that they cannot account for subjective experience and agency, amongst other things" merely on account that they are neither confirmed nor unconfirmed. I am confused about your stance. You allow yourself to have strong opinions about something unknown yet criticize other people for the same.

I think it is absolutely normal that the core of a theory is based on not directly testable assumptions. And it's normal that people push it forward if it bears fruits, that's not a fallacy in any way, that's normal inquiry that may or may not lead to successful results.