Racism plays a major role here [1,2], and it's certainly a motivation for Bill 21 at least. Remember, this is the ultimate culmination of a village writing a handbook for immigrants that included not eating babies, like they do in their home country [3].
Bill 96 is a direct violation of the Canadian Charter of Rights and Freedoms, so you're right it's not racist, just a denial of my guaranteed rights as a Canadian Citizen. Ignoring First Nations, of course, many of whom speak neither French nor English. If you do count them, then Bill 96 can also be considered racist