It really isn’t quite that simple. There are Intersex people who have ambiguous genitalia and there are several viable chromosome combos besides XX and XY.
In sexually dimorphic mammalian species sex is most commonly by the ability to produce either the small or big gametes, not chromosomes.
In humans syndromes that lead to ambiguous (“intersex”) presentation and/or chromosomes are still operating in a binary system, eh Klinefelter affects boys.
The same is true for (very) rare hermaphroditism: in this case both gametes could in theory be produced, in practice though individuals are sterile.
Those are birth defects though. For instance, we still understand humans as being bipedal despite anomalies that prevent a tiny fraction of humans from being bipedal.
`XXY` is a valid combination of sex chromosomes. The person has two `X` chromosomes so is female. The person also has one `Y` chromosome so is male. Which sex is the person? Is it a birth defect?
XXY is Klinefelter syndrome. By definition it's a condition where boys and men are born with an extra X chromosome. Their sex is male because, also by definition, the presence of a Y chromosome means male.
In humans syndromes that lead to ambiguous (“intersex”) presentation and/or chromosomes are still operating in a binary system, eh Klinefelter affects boys.
The same is true for (very) rare hermaphroditism: in this case both gametes could in theory be produced, in practice though individuals are sterile.