Of course you don't need to download the whole repository as with npm. But just the links of the optimized distributable assets. In short, your /dist folder.
Since NPM is now the predominant way of distributing packages, they don't usually have a /dist folder so individual packages would again need to think about this. This was one of the reasons that bower is not really used anymore.