I think the argument rings hollow to me mostly because it's not that Harvard has to charge this much. I'm sure they could be providing the same quality of education for 1/5th the price. In fact, with the endowments many of these schools have, they could probably go tuition-free for a couple of decades and still be fine.
They charge this much essentially because they can (govt-subsidized loans), and because it helps them maintain a certain reputation.