Right, but Abraham Lincoln did nothing wrong by going to the play and the author did something wrong (I don't mean in a moral sense, but in a sense of 'incorrect') by going to the United States without the proper visa.
It's a common expression. [1] In this context it means: this guy's treatment by the customs & border officials was so reprehensible and abusive that his legal right to enter the US is beside the point, just like how the actual merits of the play were beside the point for Mrs. Lincoln.