> The prescientific suggestion that humans derive their experience of existence from spiritual mechanisms outside the physical world has had notable social consequences, but no success as a scientific hypothesis.
Not sure what you mean by 'why' here but scientists have done a lot of experimenting with psychological experiments, neurology and the like and there hasn't been much sign of spiritual mechanisms outside the physical world. Now if seances worked better or reality was like Ghostbusters that might be counter evidence.
Depending on your definition of "outside the physical world," it's at best untestable. You can explain consciousness by saying it comes from a massless, noninteracting, odorless, tasteless... etc. but that explanation doesn't really help. So far, lots of other things that were previously explained by non-physical whims of God have had at least partially reductive explanations in science; a lot of people hope that consciousness will yield the same. It still may be possible that consciousness is a spiritual phenomenon, but with no way to test, prove, or expand on that, scientists would rather not close the book there.
The modern philosophical argument against a scientific explanation of consciousness is that science is an objective, third person pursuit, abstracting away from subjective, first person. If so, you can't hope to explain the subjective in terms of the objective.
There is no need to invoke the spiritual or supernatural to see that consciousness is a problem for science.
It honestly depends on who you ask. A lot of people think consciousness is inherently unsolvable by normal science (Chalmers' "hard problem of consciousness"), but there are plenty (Dennett etc.) who will deny that such a hard problem exists, and make reasonable arguments that the scientific method can make headway in reductively explaining consciousness.