Yes, it basically means "solve 𝐗𝛉 = y for 𝛉", for which inverting 𝐗 is one of the many methods that can be used to solve it. MATLAB also has the same backslash operator to mean the same thing.
I don't know about the pseudoinverse in particular, but at least in the case of normal inversion -- you almost never want to invert a matrix, but sometimes you do, so it should be possible to sensibly express the idea of 'invert' I think.
True, but the specific combination of inversion and multiplication, with a well-known meaning, could be transformed. Standalone inversion would still stay an inversion. Algebraic identities are definitely one thing you might want to exploit with advanced compiler macros.
I guess the only concern I'd have is, is it possible that someone knows what they are doing (so, something is special about their matrix) and wants to do an in-place inversion, multiply, and then use their inverted matrix for something else. I'm don't know Julia, though, so I'm not sure if this would be a weird thing to do. (is invert even in-place?)
Not sure about inversion, but if I remember correctly, LAPACK can do for factorizations such as LU in place. You can the refuse the factorization to solve for many RHS.
The normal way (as far as I know) to do inversion in LAPACK is to do LU (in place), and then do invert (also in place) on the LU factors (so they get eaten up by the invert). I guess it would be possible to do: LU, then do some solves, then do invert with those LU factors, it just seems a little odd.