My research suggests that 42 states contain at least one Whole Foods. The eight states excluded are (west to east) Alaska, Montana, Wyoming, North Dakota, South Dakota, West Virginia, Delaware, and Vermont. These are exactly the seven states with population less than one million in the last US Census ... and West Virginia. (Note, however, that the District of Columbia -- also with a population under one million -- has four stores.)
Although it's a strange thing to measure, approximately 7.5 million people (~2.3% of the US) live in a state without a Whole Foods.
You're welcome to look at https://www.wholefoodsmarket.com/stores/list yourself. The store you're remembering might be at 475 Wilmington West Chester Pike in Glen Mills, PA, which is plenty close to Wilmington (as the street name suggests) but is in fact ~2 miles outside of Delaware.
Although it's a strange thing to measure, approximately 7.5 million people (~2.3% of the US) live in a state without a Whole Foods.