That's an interesting argument, but not one that I agree with. Rights exist without being granted. In an unorganized society, theoretically you have all possible rights. Society limits your rights, it doesn't grant them.
It's important because you won't fight to preserve rights you don't think you possess, and there is no conflict if the government decides to take control of something if you don't have an acknowledged right to it given by the government.
Essentially, in the scenario where you only have the rights the government grants you, the default uncharted behavior is illegal or immoral. In a scenario where you have all the rights not limited by the government, you can do new things and have the right to engage in new behavior without fear.