One could say the same thing about death (or life). Once you’re born, death is the only “forward” that exists. You can’t calculate its exact distance but it’s inevitable.
And seeing as how time is only something observed at the macro level and is still completely unexplainable scientifically, you're really hitting the nail on the head here.