An important detail to add to that, to ensure a complete picture is created of the situation, is that originally the atol was uninhabited. The French created a colony there of 60 or so people and slaves.
Umm, "people and slaves"? Do you really consider slaves to not be people?
Also, originally Iceland was uninhabited. The Norse created a colony there of - as you put it - "people and slaves". That equivalent history surely does not affect the legal status of the descendants of those people now living in Iceland. Why should it affect the people now living in the BIOT?
In the interest of completeness both Great Britain and North America were also once uninhabited.