[mathw]

Or we live in the timeline that leads to the invention of time travel, not the one the time travellers are visiting. Maybe the universe allows time travel but it always creates a divergent timeline when you use it.