I think you are onto something personally. Many in the UK had an inflated view of the importance / strength of the country harking back to the empire (admittedly final days pre WWII). Which was in fact more todo with to being close to the US who propped the country up and kept it at the top table, because we would always agree as opposed to the actual merit of the country.
That combined with an island mentality and a fear mongering, sensationalist press.
Case in point, my mother complaining of immigration, although she lives in an out of the way town where they had never seen a brown person (apart from at the hospital in the next town) until probably the late 90s.
Now she insists there are too many eastern Europeans, however she swears blind she would never get an Englishman builder / plumber because the Polish ones (catchall, means any Eastern euro) are cheaper, more reliable and just better.
But she wants to make Britain "great" again, "like it used to be". Pressing her when, she cannot answer. Asked if we should invade a "brown" country and get rich of plundering their resources and ruling via the military like we used to do, "no no no we shouldn't do that". "err mother you do realise why we used to be great and rich don't you"
An Anglo free-migratory superstate combining Canada, NZ and AUS has been proposed to fill the shortage of labour once your continental plumbers leave.
As an Aussie, I have no desire to return to the island my ancestors fled 160 years ago but if it enables me to retire to a piece of land in Quebec or the Maritimes, I'm all for it!