Some states in the US with large numbers of EV drivers are already kinda doing this. I now have to pay a flat EV tax on my registration, although it's still less than I'd pay annually in gas tax if I drove an ICE car.
And I don't believe any states manage to cover the full cost of just their infrastructure and upkeep through use taxes (gas tax, registration, tolls). Non-drivers still end up subsidizing the remainder.