That wasn't my experience when I was there. While it's true that the average citizen may/will not speak English, it's more common than just in the high end areas that you mentioned.
I was in HK in 2013 for Chinese New Year. Lots of foreigners working there, especially Indians and Australians, but the local population struggled with English. I was in Singapore in 2010. Everyone there spoke fluent English.