[RagnarD]

"This AI child knows about the world more than we do since it has been trained on the whole internet, but it doesn’t have millions of years of evolution, genes, or a nervous system to back up its morality and empathy. This means we need to install morality in AI from scratch. But how do we install something in a software system that we can’t even define ourselves?"

Two things jump out to me. First, there's a vast amount of information about morality in that internet-scale data dump. It's not some mystical thing orthogonal to knowledge, which ties directly into the second thing: maybe the author can't define it, but many others can and do, especially philosophers, though YMMV given which philosophy and philosopher.

viz: "What is morality, or ethics? It is a code of values to guide man's choices and actions—the choices and actions that determine the purpose and the course of his life. Ethics, as a science, deals with discovering and defining such a code."

There are different systems of ethics, and those themselves can be reasoned about. We should want one that values rational human life. And among that vast data dump, there's no doubt that those ethical ideas have been extensively written about - so it can be used as a reference point to actively emphasize and select the ideas.