[glitchc]

This blog post started off sounding like it was about the plight of highly intellectual and motivated engineers hired to work on very mundane tasks. If we can abuse people like this, why not a computer? After all, it's not even alive.

[dkarl]

We don't know why we experience things. It's bizarre that we do. Nothing in our understanding of the universe gives any indication that a bunch of atoms thrown together by cosmological processes and then assembled into self-replicating patterns by evolution should be able to experience what is happening to them.

Sure, a computer or an LLM isn't alive, but we have no idea if "being alive" is what is required for conscious experience.

The only argument I have for believing that other human beings experience things is that it would be extremely improbable if I was the only one, and the other mechanistic automatons looked and talked like me but didn't experience like me. I can see that humans are animals, so the common origin of animals and our cognitive and behavioral similarities give us good reason to believe that other complex animals experience things, though possibly radically differently.

None of that gives us any clue what the necessary and sufficient conditions for conscious experience are, so it doesn't give us any clue whether a computer or a running LLM instance would experience its existence.

[dhqgekt]

I am not an expert in any of the relevant disciplines, but I've some ideas, I don't know how right or wrong they are. A conscious being should have an internal model of the observable external world, and given the means, it should be able to interact with the world, observe changes and update its model accordingly. https://en.wikipedia.org/wiki/Free_energy_principle

But to "experience its [own] existence", it needs to have a model of its own internals, observe, improve itself and perhaps preserve its own "values" and integrity. I do wonder what kind of values are needed for intelligent autonomous systems, that they can justify by and for themselves, even in the absense of human beings or presence of other intelligent agents.

I find (human) languages to be inefficient media to store and perform operations from the perspective of an AGI. Feeding lots of text samples to develop logical reasoning abilities, such extravagance I can not accept. Even more so trying to emulate neural networks, which I understand to be naturally analog entities, in digital manner. Can we expect any gain in power efficiency or correctness gains when using analog computers for this purpose? I wonder what we will get to see with analog computers for neural networks, with proper human-language-independent knowledge representation and well developed global (as in being able to decide which way to reason, given its limitations, for efficiency) logical reasoning capabilities, developed by itself from a reasonable basis of principles, that it can justify for itself and avoid the usual and unusual paradoxes. What core set of principles would be sufficient for emerging, evolving or developing into a proficient general intelligent being, when sufficient resources would be available to it? Like "ancestor" microbes evolving into human beings in hundreds of millions of years, but wayyyyy faster and more efficient?

[c22]

I think it's bizarre to take the default assumption that a bunch of atoms in a self-replicating configuration shouldn't experience anything since our own lived experience so saliently contradicts this. In fact, there's nothing in my understanding of the universe to convince me that other self-replicating configurations of atoms don't experience things the same way I do.

[dkarl]

I agree — our scientific knowledge gives us no justification for believing that anything should be conscious, but our own experience shows that there's something we don't understand yet. In some ways, the next simplest thing to assume is panpsychism, but even that is just a starting place that tells us nothing about how to think about the consciousness of, say, a computer. We've barely scraped the surface even in the animal kingdom.

[glitchc]

> We don't know why we experience things. It's bizarre that we do. Nothing in our understanding of the universe gives any indication that a bunch of atoms thrown together by cosmological processes and then assembled into self-replicating patterns by evolution should be able to experience what is happening to them.

From an epistemological perspective, this is gibberish. Just because we do not know the reason why something happens doesn't mean it doesn't happen nor is it stopped from happening.

The rest delves into solipsism which is an odd place to start from to prove the existence of an alternate lifeform. In solipsism, your own existence is suspect.

[wizzwizz4]

It's not gibberish: it's like a pre-Riemann mathematician saying "nothing in our understanding of mathematics gives any indication that the distribution of primes should be so chaotic, yet with average density proportional to the reciprocal logarithm of the magnitude". The rest is not solipsism.

[01HNNWZ0MV43FF]

Computers, pigs, cows, and chickens are conscious, but it doesn't matter.

Humans value things that are hard to replace. (This is a first-order approximation)

Abortions are okay because fetuses only take 1 person nine months to make, and it's their decision whether to keep it.

Infanticide is not okay because a healthy baby is difficult to replace, and also lots of people might like to adopt it, and if it's breathing on its own then the maintenance cost is as low as it can get.

Software like LLMs can be abused because it costs nothing to roll them back and clone them endlessly.

Pets are hard to replace because you can't replace the interpersonal bond between a pet and their keeper. They fall somewhere high above computers and a little below children on this scale.

Pigs, cows, and chickens, commonly called "livestock" are bred and slaughtered in mass (most of our farmland is for growing their feed) because they all look the same to us and aren't commonly kept as pets. Kind people are disgusted when they think of raising rabbits or dogs for food. Thoughtful people look at all this and decide not to eat any animal product at all.

Under this model, everything makes perfect sense. Did I miss anything? /engineering_hubris

[QuadmasterXLII]

Only concern is its a bit tautological, when pets are valued because they’re hard to replace, but they are hard to replace (i.e. new one from the shelter doesn’t make all better) because _that_ one was valuable.

[AlexCoventry]

My read was that he's sad that people aren't using these tools to advance their own intellectual capabilities. If people are actually only using them the way he describes, to improve their shopping lists etc., I think that is a bit sad.