The opposite of an immigrant is an emigrant (though that does depends on the observer's location).
I think expat assumes a temporary stay outside the country, where immigration/emigration is maybe considered permanent? I also think expat generally indicate a higher educated job, so do feel like it is a way to distinguish themselves from other forms of migration.
I have always thought that expat refers to those who maintain their citizenship of country of origin (an American citizen living in Spain) and an immigrant is someone who is either a citizen of the destination country or on the path to become one.
Your definition of an expat matches undocumented immigrants from Latin America to the US perfectly, and yet somehow the word is never used in that context.