Hah, amusingly I failed to consider that. I wonder how fast a sun would have to move through to have a negligible (or at least not catastrophic) effect on orbits?
The Oort cloud around the Solar system ranges halfway to the next star. That far out the Sun is probably not the only relevant influence anymore. Even though stars would have to get really close to be able to mess up orbits of the inner Solar system, a large comet can absolutely wreck the biosphere if it hits us.