[seabee]

This kinda happens already, or at least it used to many years ago; you have to be an active user for your vote to count.

The most obvious effect is movies that drop in the rankings 30 days after release; people who just saw a movie give it 10/10 and then stop participating, so 30 days later they are no longer considered 'active' and their vote stops contributing to the overall ranking.