[logicprog]

> People don't play dungeons and dragons anymore

You sure about that?

Also, the examples you're giving aren't remotely analogous to AI-generated letters, because your examples are about the means of communication — the medium and presentation — changing, but the underlying meaningful content being communicated is still fundamentally generated by human beings, whereas with AI generated letters there's no underlying meaning or intentionality at all, because no human generated it. So it isn't just the medium that's being changed fundamentally this time, but what's being communicated with it. And we already have an equivalent of AI-generated letters: those premade cards for all occasions you can find at grocery stores. And sure enough those cards aren't valued that highly in comparison to handwritten notes or talking in person

[jart]

An example that's more analogous to how you're feeling would be the many great crises of faith that happened in the course of modernity, due to science unriddling nature. What you're telling me is that the human mind has a magic touch which imparts meaning and legitimacy to the things it creates. How different is that really, compared to our ancestors, who saw magic in the forces that governed the natural world around them? Some people saw the celestial bodies as spirits or even gods who were powerful and overlooked humanity. It was a rude awakening for many of them to discover that some scientist in his garage could predict and explain their movements. To learn the special roles we'd imagined for ourselves actually had more in common with the things we looked down upon. I think we're at a similar moment today because science has finally produced a theory of consciousness that's compelling, due to how it lets us recreate it.

[logicprog]

That's a straw man. I'm not insisting there's anything magical or spiritual or metaphysically special about consciousness. I am merely insisting that consciousness is a particularly complex recursive metaprocess that is utterly absent from what GPT is doing. Not that it couldn't theoretically be replicated by a very large and complex neural network model. Just that it is abundantly clear that the sort of thing GPT is doing and the sort of thing we do are simply not the same. It does not have meaning or intentionality and it does not have consciousness. You just need to spend a little time actually looking at the utter drivel that it produces when you ask it things or tell it to output things to realize that not only is it highly formulaic, but it does not have any underlying understanding of the concepts or principles or facts or intention of what it is doing, it is merely producing an assemblage of words that it has been trained to deem fairly convincing.

The stochastic parrot model that GPT follows is not even remotely a convincing theory of consciousness, whatsoever, and it has certainly not let us reproduce consciousness. I can hardly even believe you are saying that it has. That's like saying traditional computer game graphics rendering prior to raytracing becoming mainstream is a "compelling theory" of how physical light actually works because some game looks almost photo realistic. In this analogy, I'm the one that's pointing out that what physical light does in the real world is extremely different from what traditional computer game graphics does and even if you can get it to be superficially close it will never be as powerful as the real thing, and to do that we would need a much more computationally expensive ray tracing model. I'm not saying there's anything inherently unique about human brains or that you can't model Consciousness with the computer I'm saying we haven't gotten there yet.

[jart]

But it is though. You could learn a lot about physics by reading video game source code. I'm not saying that understanding a thing well enough to emulate it implies mastery over the thing itself. For example my Blink emulator that I wrote does a pretty good job passing as an x86-64 Linux computer, and I think that makes it x86-64-linux enough, even though I don't necessarily understand how Intel's foundries work. I'm not saying video games and GPT are a theory of everything, but if it acts consciously enough to fool us into thinking that it's conscious, then I think that makes it conscious enough, which would mean that the ideas which went into building it reflect the deepest understanding of consciousness we've got.

[logicprog]

It doesn't act conscious enough to fool us though. It doesn't seem remotely conscious even at first blush, and seems less so the more you interact with it. And I think the stochastic parrot "model" of consciousness is very obviously not a good model for understanding the processes going on in the human brain, because it leaves out so many aspects: the fact that self-awareness and metacognition are a key component of consciousness, the fact that humans are trained as they grow up by experimenting with an environment that rewards understanding consistent principles and concepts, instead of just being rewarded for stringing together vaguely convincing-sounding sentences, among others. Like, just because it can create something vaguely superficially similar doesn't mean it's a good model for understanding consciousness at all. That's not how this works.

[jart]

Are you saying the Turing test wasn't passed? If you think you understand machine learning well enough to dismiss it as a science because you know a buzzword from a feminist critique that also came out of Google Brain, then I'd say their inclusivity initiative certainly did its job. The reason it passes the Turing test is because the rest of us can't tell it apart from people like you.

[logicprog]

I have to ask, are you seriously claiming that stuff like ChatGPT appears conscious to you? Do you really think we've created consciousness?