This is possible but extremely unlikely, hence an unscientific assumption. You always assume the likely. That is is more likely that you are in the middle.
You could argue that ONE of the steps to intelligent life is unlikely, incredible more unlikely as we assume. But the questions remains: Why are they not here? I am afraid we may not like the answer, whatever it is.
The answer is obvious to me: the chances of life are very small. The universe is very big (that's some underestimation). So it is likely, that we are not alone in the universe, but we may be so far apart in terms of space and time that there will be no contact ever.
Just take a look how far our signals have travelled since we began transmitting — they only covered a small part of our own galaxy.
https://en.wikipedia.org/wiki/Fermi_paradox
Related statistics: Doomsday argument German Tank Problem Capture Recapture