The scoreboard is burnt-in by the sports channel carrying the game; to the best of my knowledge, there's no way to get the raw switched video feed before the graphics crew touches it.
They leave the scoreboard because it contains branding for the host team / stadium and frequently the local affiliate actually running the broadcast. NFL Gamepass, and both NHL and MLB live air the original broadcast feed.