Yes, that is the core of the criticisms of meritocracy. Who decides what's worth merit? The ones who have already accomplished certain ends decide that those ends are meritorious.
Indeed, although, if we don't generalize too broadly, I prefer it this way. I don't want to be obligated to follow / hire people without any accomplishments, just because the people without accomplishments decide it isn't fair to them
Everyone has accomplishments. Problems arise when people the people deciding that some accomplishments are better than others overvalue the ones they have and undervalue the ones they don't. Remember when Google only hired people with college degrees? Notice how companies devalue degrees from HBUs? It's a farce to pretend that these are objective measures instead of just ego-stroking.