It's a problem that many people aren't aware of, and it's a very compelling argument. The eventual goal should absolutely be to completely ignore irrelevant factors (gender, race, etc) when evaluating anything (such as a conference proposal), and evaluate only based on merit; the question is over the best way to get to that point, and there's quite a bit of evidence to suggest that attempting to ignore it right now doesn't actually work.
The concept of evaluating exclusively merit is completely sensible. Doing so in practice is not trivial. And there are indeed published studies showing that attempting to do so not only fails but produces more bias.
The concept of evaluating exclusively merit is completely sensible. Doing so in practice is not trivial. And there are indeed published studies showing that attempting to do so not only fails but produces more bias.