Distance is usually the wrong measure in space. Something like delta-v will give you a much better scaling as once you manage to get something to orbit the rest is actually a lot closer than it would seem on the ground.
Not to say the effort somehow becomes peanuts, cheap, or easy... but the jump in delta-v needed to go from "100 km vertical ascent" to "hit the moon 350,000 km away" is more like a ~6-7x increase than a 3,500x one. If the moon were instead 700,000 km away the factor would still be ~6-7x.
Everything you've said is correct, but Delta-V scales logarithmicly with fuel load - you need to carry the new fuel. So for purpose of discussing altitude (a valid way to look at getting to the moon) the size of the rocket, and the fuel expended, does in fact grow much closer to linearly.
What I actually started with was comparing Electron to the current bos.space rocket and seeing the relationship was nowhere near linear. The above is the largest component of why I could think of but there is always more than 1 thing going in.
Not to say the effort somehow becomes peanuts, cheap, or easy... but the jump in delta-v needed to go from "100 km vertical ascent" to "hit the moon 350,000 km away" is more like a ~6-7x increase than a 3,500x one. If the moon were instead 700,000 km away the factor would still be ~6-7x.
Cool site for delta-v estimates https://deltavmap.github.io/