CLAs are always a good idea if one is maintaining an open source project. I'm not going to track down every single contributor who made some small PR perhaps, just to make a license change.
The license could change for any number of reasons, making a paid version, making it GPL, etc. With a CLA, I don't have to assume, I'm able to change it regardless, that's why for maintainers they are useful.
Keeping the software free, i.e. not proprietary, even from maintainers trying to make it so, is the very point of GPL.
When you ask contributors to sign a CLA, you are saying that you might want make it proprietary in the future and reserve the ability to do so. You should never sign one if you don't want to contribute to proprietary software.