That's why Musk had art of BFR sitting on a Jovian moon, for the same delta v to get into earth orbit, you can get anywhere else in the solar system, quick (relatively). Big rockets and large mass fractions enable all that, as my comment above indicates. Rocket equation is for real, but so are economies of scale.
There's some opinion there, but personally I would still say yes. There's a lot hiding in the "relatively" in le-mark's sibling comment.
We really need to be able to get places in a couple of weeks, and chemical rocketry just gets absurd for that; even if you can do it, you've got to fling massive refueling stations around and such. A fusion-based system would have power density sufficient to not just fly to Mars in a reasonable period of time, but to fly back without having to be refueled, which means we can go places without first figuring out how to build a fairly expensive and massive infrastructure that allows us to get back.
Even if you use fusion power you need to use some mass that you can accelerate and then expel opposite to your direction of motion. So while you would need less fuel, this is only really true if you also have what would amount to a miniaturized particle accelerator beyond anything we are able to build so far. Or put other words simply producing lots of energy is not enough.