"Hey, wait a minute, wouldn't this time and energy be better spent helping all disadvantaged people instead of one small group to the exclusion of others?"
..is a criticism, not a "shooting down". Being altruistic does not grant you a blanket exception from criticism.
"You didn't help everybody" is not a valid criticism of an initiative to help some people.
If you have some other people you want helped, then help them yourself rather than shooting down other people's altruism.