I'm normally on board with variations of this argument for things that don't significantly affect other people, but cars very much affect other people. Larger cars are more likely to kill other drivers, and especially bikers and pedestrians. The damage they do to the road is also a LOT more [1] so people who drive smaller vehicles and/or walk/bike are effectively subsidizing people who drive larger vehicles. They also cause more emissions meaning more medical issues and accelerated climate change.
heavier cars are deadlier in accidents, they cause more damage to road surfaces, they require wider roads and parking spaces which consumes land, they require more resources to produce, they require more energy to operate, they produce more pollution, and so-on
Because cars have an impact on all of us, not just those driving them. These impacts range from tens of thousands of auto deaths every year to trillions of dollars in environmental damage that we will need to clean up eventually. Your libertarian fantasies are a poor match for the actual world.
[1]: https://en.wikipedia.org/wiki/Fourth_power_law