Exceptions exist for private clubs according to https://en.wikipedia.org/wiki/Civil_Rights_Act_of_1964 but with regard to businesses getting to decide which customers they serve, it looks kinda like segregation on the basis of race (or gender, religion, or nationality) was outlawed.