You're missing the point. No one is being "punished" for their ancestors having owned slaves. The idea is to assist people whose ancestors were exploited, legally and systemically.
It may help to think of it as assistance for victims of a natural disaster, which seems pretty universally acceptable. The disaster being a very messed up society. Or as something like the Marshall Plan that helped restore Europe after, again, society’s global screwup.