Well, I've hit birds with wingspans around a dozen feet (didn't go measure). They make a mess, and have killed people I knew, but that was mostly because of their size and the penetration of the windscreen. Hitting a smaller object could certainly cause some damage but it wouldn't likely be immediate and catastrophic unless it was in a critical spot.
Well, sure you don't want to hit anything unless you want to hit it, but doing so is unlikely to be the mushroom cloud event that seems to be believed occurs when hitting a drone.