[NeedMoreTea]

Not at all. Where you solve the transportation of waste and getting in water through the river, only the most upstream city was getting potable water, downstream was after washing and waste went in. Wells frequently went down to a polluted or at least unsafe water table. Wells out of the towns and cities were generally safer. Few if any towns were the sole town on a river.

Tea was initially an elite drink in China, the expense in part explaining the rise of the rituals and ceremonies (and lots of equipment) surrounding it. Much the same happened with coffee.

Rome was indeed an exception thanks to their development and aqueducts, but that potable water was coming from well outside the city. In the outlying regions of the empire, without that infrastructure, wine seems to have been the usual, and water for bathing.

Remote enough is akin to upstream enough -- with no shite, washing and other pollutants, getting potable water from a remote spring or well is perfectly feasible.