According to Dominic Cummings (who ran Vote Leave) it was absolutely the case that immigration was the No 1 driver for the vote to leave the EU. So much so that they had to explicitly set aside 15 minutes at the beginning of every focus group to let people rant about immigration before they could get them to move on to anything else at all.
Anecdotally, this is also what my parents found on the doorstep.
The sad fact there being that immigration being an issue rather than a net benefit itself was a distraction perpetuated by the Murdoch press.
Fear of 'the other' is a timeless scapegoating tactic when you really want to distract people from the real causes of their problems.
In the case of the UK those problems are entrenched and systemic classism and privilege, a financial services industry rife with fraud and cowboy behaviour and our own sycophantic career politicians that compared to the EU, have done very little for the working poor and other disadvantaged demographics in society.
Anecdotally, this is also what my parents found on the doorstep.