If both keys exist, then it’s an even bigger signal of user mistake. Are you going to fail then? Distinction becomes pretty arbitrary.
As a user, I’d rather have a failure at boot time if I got it wrong. Just because I had a typo once doesn’t mean I always make it. What if I tried to transform my own config? I’d have to follow special case your referers property like you did to keep it consistent.