There's no need to do purely sequential download, though. You only need to download sequentially at the speed of playback, the rest of your bandwidth can go to the rarest chunks.
You're right, but then you'd need to programmatically determine the video bitrate in the torrent client, or integrate it with the video player. Either way is a pretty heavyweight solution.