Many people who assert things about James Damore's memo are engaging in the same dishonest anti-science tactics which Climate Deniers and Evolution Deniers engage in. Science is not a democracy. People also don't get to impute James Damore's motives just because they feel strongly about it.
Here is a reasonable and cogent critique of the Damore memo. Liana K makes a very good argument that James Damore is wrong. However, note how many general falsehoods about the memo she calls out.