Not everything in politics is pure logic, there are lots of emotions too. Not aligning with neo-nazis is one of those emotions.