[gavanwoolery]

You are exactly right. This type of argument is called "ad hominem" (translation, appropriately, is "to the man"), and is the basis of many fallacies and Republican/Democrat debates.