Once again, depends on who you ask.
By international laws (Universal Declaration of Human Rights) it has been illegal starting 19481[1].
USA, however, still struggled with racial segregation until the 1960s and until then wasn't really illegal in the States. In South Africa, it probably wasn't illegal either as the government made the law themselves.
[1]Or 1976