You're right. You wouldn't care about everything in npm. The point is this: When using Gulp/Grunt, we have to search for a plugin. This greatly limits our selection.
I see your point, it makes sense. But I have to note that in gulp (at least) you aren't restricted to plug-ins. You can use "generic" npm packages, as long as they output a stream (e.g. browserify).