When we have completed a todo task we "cross it" to mark it done. i would say the x to close is intended to represent a "crossing out" not the letter x. It is pressed to signify a task has been completed.
My thoughts exactly. Not being from exactly a Western culture (I'm Russian) I always saw the X not as a letter but rather as a cross (Russians say "click the cross to close the window"), a sign of deleting or cancelling something by crossing it out. It made perfect sense so I never even thought there could be other explanations. All because in our language there is no X-exit connection.