[khafra]

Rationality can serve whatever values you have. You can rationally optimize the amount of love, happiness, and fuzzy puppies in the world if you want to. You can also rationally strive to keep a company efficient and non-evil. But the bigger it gets, the harder that gets, modulo economies of scale.

[thwest]

Agreed. At certain scales, it no longer makes sense to give primary mover status to humans within institutions. It is important to realize that there are several mechanisms within institutions that alter the values they serve. It makes more sense to treat them as black boxes and reason about outcomes rather than intentions.