Easy solution. Turn 180 degrees. Ask the coworker you are now facing to change the desk with you. Problem solved.