If you have a nicer mouse with a quality sensor, a pretty slight bump or vibration to the desk can cause it to move and wake the computer. Try disabling wake on mouse so you can only wake with the keyboard.
Yup. I puzzled over this one for a while when it happened to me. My mouse was on a pull-out surface from the desk and thus any vibration was amplified. It was sensitive enough to wake the computer merely from walking up to it.