Not to mention the letter X sounds like the word "exit," or "ex it." In this way using X as a keyboard command to represent exit might be distinct from the way OP posits X to represent batsu (false, bad, wrong or attack).
though X already means false or bad in western culture, the opposite of a tick (✓). there's also "ex-partner" (my ex) etc. in a gui, it's an ex-window.