I'm a big fan of the "veil of ignorance" philosophical thought experiment[1]. Let's say you are given the freedom to design a society and its moral code. Then you will be born into that society and subject to it. The trick is you don't know who you will born into. It's a roll of a dice. What kind of society would you design such that that seems like a winning game?
I'm fairly certain that when living in that society, you would wish to feel valued by others and that others believe you deserve a certain level of dignity and respect. Since you don't know who you will be born into, that suggests that you want a society where everyone is valued and granted dignity. That in turn extends even to people who are unfortunately not able to produce material objects with a level of skill superior to what technology can produce at some specific moment in time.
If you agree with that, then we should advocate for granting all people value and dignity "just because" and not because they happen to be better at producing bytes than an LLM. That way, even if the LLM gets better at producing those bytes, you are still valued.
That line doesn't really work, since I'm not Sam Beckett and I won't suddenly Quantum Leap into anybody's body. Why shouldn't I take an asymmetric attitude? These kind of arguments try to derive impersonal values by starting from principles of self-interest. But there's no need, because self-interest is impersonal values, in disguise. Maybe you value getting personally improved in one way or another, like getting richer or more popular or stronger or acquiring offspring, but we can play the "why" game on that until it turns into some motivating abstract impersonal goody-goody value such as "I value people".
I mean, why do you wish to feel valued by others and get respect? Is it just so that they stay out of your way and give you things? What are you going to do next, with the space and the things? This assumption, that you care what they think of you, pretty much begs the question about valuing them back. What you value is what you want to do, and the other people are involved in that, and asymmetry doesn't help get it done, except very locally (that's privacy).
Personally I often wish that all the people would go away, but I value creating knowledge, and I have to admit that I'd be crap at doing that on my own, so fine, I value people really, you can all stay.
> I mean, why do you wish to feel valued by others and get respect?
I believe we have evolved to be a social species whose brain innately rewards prosocial behavior. Your question is akin to asking why cats chase mice or why birds fly.
What a strange world - I'm an atheist, and the quoted argument resonates intensely with me, as somebody who believes in the innate value of humans even in the face of objectivity.
If you believe in the objective truth of the quoted source, then that makes the claim that humans are valuable about as robust as any claim can be.
If you do not believe in the objective truth of the quoted source, you must still reckon with the fact that this is the (start of) the punchline of the opening narrative of the foundational text of the world's largest religion or religion family. (The punchline continues with the judgment that the universe, being merely good before humanity, is now very good with humanity.) That is to say, this is an incredibly widely and deeply held value by a vast number of people over millenia, not to mention the many others whose religions may contain analogous claims.
Remember, the statement "X is valuable" is always shorthand for "X is valuable to Y". In this case, at absolute worst we mean "humans are valuable to humans," if not also "humans are valuable to God".
That's all very well but prove it first. "Narratives" are perfectly fine stories but not physical truths. It is absurd to cite a "narrative" without proving it's physical truth first.
> That is to say, this is an incredibly widely and deeply held value by a vast number of people over millenia, not to mention the many others whose religions may contain analogous claims.
I agree. The evidence for that claim is weak and if it is cited to be the basis of the first claim the likelihood of the rest forming an accurate model is low. Thank you for the comment. I might have otherwise wasted more time on the linked post than I did.
I'm a big fan of the "veil of ignorance" philosophical thought experiment[1]. Let's say you are given the freedom to design a society and its moral code. Then you will be born into that society and subject to it. The trick is you don't know who you will born into. It's a roll of a dice. What kind of society would you design such that that seems like a winning game?
I'm fairly certain that when living in that society, you would wish to feel valued by others and that others believe you deserve a certain level of dignity and respect. Since you don't know who you will be born into, that suggests that you want a society where everyone is valued and granted dignity. That in turn extends even to people who are unfortunately not able to produce material objects with a level of skill superior to what technology can produce at some specific moment in time.
If you agree with that, then we should advocate for granting all people value and dignity "just because" and not because they happen to be better at producing bytes than an LLM. That way, even if the LLM gets better at producing those bytes, you are still valued.
[1]: https://en.wikipedia.org/wiki/Original_position