Looking at this from another angle: even though our self-interest is strong, we live the majority of our lives under social rules that promote virtue (or rather, virtue is defined by the social rules). The number of humans who can actually pursue their self-interest in any real way is limited. Most of us have to work for many years, and surrender most of our lives to the broader world.
In fact, true psychopaths never seem to occupy more than 1% of the population. They are the self-interest pros because they do not feel fear or stress the same way we do and can make the most of it. But at the same time they find themselves limited by society even if a few of them succeed enormously.
All this to say, I'm not sure virtue is that limited. These are competing forces that have reached an equilibrium but most of the fabric of our lives IS virtuous.
Virtuosity tends to extend less than self-interest, which seems to be approximately boundless.
Having said that, I guess ones perspective on that particular duality is Hugo influenced by ones mood in any given moment ;)