Anything we evolve to desire signals genetic fitness.
Genetic fitness means anything that results in more grandchildren in the end. Physical health is a big factor, sure, but so is intelligence and social skills.
Maybe this is in part a linguistic disagreement. By "attractive" I don't just mean "looks good in a photo", but any factor that attracts a partner.