Agreed. In addition to "codes of conduct" that seem to assume all attendees are lecherous, racist rapists some conferences step outside their focus and end up turning away potential attendees. For example, this military member who really seems to be the target market for DjangoCon but who can't attend due to past pro-marijuana speakers. https://www.reddit.com/r/django/comments/4dihy4/django_train...
While I suppose there may be some codes of conduct out there that truly make those assumptions, most of the ones I've seen are simply predicated on the assumption that it's better to proactively make rules outlining acceptable behavior -- and, ideally, outlining both the ways complaints will be handled, including enforcement.
We'd all like to believe that "trust everyone to not to be jerks to one another" is code enough for any convention, and I get that "X, Y and Z will not be tolerated" can come across like taking sides in a debate. But establishing a CoC first is the social engineering equivalent of test-driven development. Handling complaints on an entirely ad hoc, subjective basis works as long as complaints remain relatively minor, but the failure mode can be pretty spectacular, and not in a good way.