[jfengel]

Then we'd have a different set of observations that conflict. We know the age of the universe from several different directions: the rate at which distant galaxies move, the temperature of the cosmic microwave background, the temperature of white dwarfs, etc.

These agree to within a relatively small range. If the number were substantially different, it would imply that something deeply fundamental (and probably several deeply fundamental things) was wrong.

It's much more likely that our understanding of galaxy formation is wrong. That's much less fundamental, and much harder to observe.

It's just like debugging code. You start with the obvious stuff. It's much more likely that the error is in your program, for example, and not in the compiler. That's not proof, but you'd be foolish to start anywhere else.