"over 60 percent of the Forbes richest 400 Americans grew up in substantial privilege" (wikipedia) and add in race and gender rigging of the system and IQ is a factor among others.
Wealth can only accumulate exponentially, and there are big limitations for the "growth" factor. You have to build on the wealth of your parents. This makes it substantially easier for those with richer parents!