Yeah there is no objective baseline. However, if the Agent could evolve a language that averages the biases of all human, that could be used as the baseline couldn't it?
Some humans can not make mouth sounds; some humans don't have arms.
You already can't hit that baseline with one language.
As somebody who has done a great deal of idle bloviation about ideal languages, I can say that there are many obstacles. For example, you need to add a true noun because a new material is discovered and it's getting very popular. You want a short and efficient name for it, but all the space is allocated. So you add it to your table of weights, and suddenly every prefix-coded word is pronounced and written completely differently.
If you can't use prefix trees, how will you allocate the words efficiently?