They transferred via Sun's acquisition of the Strongtalk team (that also got them Gilad Bracha). Strongtalk was also one of the first hybrid typed languages (like Dart and Typescript).
Much of the innovation in dynamic language run-times today also uses ideas from Self. Most of the Self people are working at Google today in various high positions.
I always wonder if the Self people felt bad about their VM efforts going into Java. Java has really been one of the least inventive and progressive languages.
I doubt they would feel sad about their technology being adopted by a mainstream language used by millions of developers vs. a research experimental language that many outside of research haven't even heard of before.
The number of programmers back then was much less than now, but even then I don't think Smalltalk was a top 10 language. I guess it depends on how you define mainstream, Smalltalk has/had its niches, some of which are still active. But there is nothing wrong with using research languages like self (which was never mainstream) to explore and develop new tech that can be applied to mainstream languages.
Much of the innovation in dynamic language run-times today also uses ideas from Self. Most of the Self people are working at Google today in various high positions.