The Tenth Amendment's Reserved Powers clause also suggests this is a states issue, unless you can contrive it to be part of the census.