The TLDR is that most of the US is single tracked and the freight trains are bigger than the passenger train has to wait even if it has priority because the freight train is too big to wait.
Also, because Wall Street is so obsessed with making the balance sheets as efficient as possible, they have pretty much no interest in capital improvements like longer sidings or additional tracks.
It's worse than that. They have a specific interest in not having longer sidings. The companies that own the tracks own the freight trains, so longer sidings would be worse since then their trains would have to yield.