Yes, almost nowhere in Central America would you (or locals) drink water from the tap. Things like brushing your teeth are a grey area.