Although with binary numbers you'd probably want to modify the rules to avoid rolling so many d20s, since you'd need to re-roll them 12/32=37.5% of the time
You can do it with two bags. Place one piece of each colour, except for pawns, in bag A. Place the pawns and remaining pieces in bag B. There’s ten unique piece in bag A and bag B produces colour with equal probability, so there are twenty distinct outcomes which all have equal odds.