SV is significantly overrepresented with east and south Asians, compared to the general US public. If you wanted to make SV match the general US demographics, they'd have to hire more white people.
Just because black people specifically are underrepresented in SV doesn't mean that it's (racially) non-diverse.
More than that, part of silicon valley's diversity is (iirc) an unusually high representation of asian men; an assumption of whiteness wouldn't get you far.
Your remark, on the other hand, not so much.