I agree. There is a clear gender difference in seeing color [1][2][3][4]. The use of colors here would invalidate the test for autism, at least in the context of gender difference, since it would be testing for how well you see shapes versus colors.
1: http://jov.arvojournals.org/article.aspx?articleid=2191999 2: https://www.livescience.com/22894-men-and-women-see-things-d... 3: https://www.colormatters.com/color-symbolism/gender-differen... 4: http://news.nationalgeographic.com/news/2012/09/120907-men-w...