There are other factors that stop the kind of life we see on earth from appearing earlier in the universe’s history. For example the first stars (the pure hydrogen ones) could not have had planets that supported life and they burned too hot and quick to reasonably do so.