Firefox is low on browser count but it's still around 4%[0]. That's enough that there will be lots of collisions. Even a small percent of a very large number is a very large number
However, if you're trying to search for somebody, and you're able to eliminate 96% of the data, you're in a much better position to accomplish your goal.
Whether or not you should care about this depends on what kind of tracking threats you're trying to avoid.
I mean yes and no. Raw numbers still matter. It's all about context. If you have a billion visitors and rule out 96% of them, sure, searching 4 million it's easier but it's still such a large number that that alone isn't enough. That's all I'm trying to say.
However, if you're trying to search for somebody, and you're able to eliminate 96% of the data, you're in a much better position to accomplish your goal.
Whether or not you should care about this depends on what kind of tracking threats you're trying to avoid.