I don't use Google analytics but I have seen time and again vocal users who seem ignorant of their own usage of an application...and very much not representative of the majority.
People are bad at analysing metrics, too. You make a change and users spend half as much time on a page. Did you make the page twice as easy to use, or so bad that they gave up?
I honestly think that a simple “we are considering removing feature X, please let us know if you’re still using this” can work extremely well. Everyone who cares about the feature will be sure to write in.
Plus, people often don't give valuable feedback when asked questions about features they want and use... people are poor judges of what they actually want, and will list things they think they care about and then end up never using or which don't affect their choices.
We do. We have real customers do real surveys of our real web sites. In person. We even do shadowing to see how real people use our sites in their daily work.
It's uncommon in SV due to laziness and an unwillingness to talk to actual human beings. Which is dumb because there are companies that will handle this for you.
But SV is stuck in this mindset that everything can be solved by an algorithm. It can't. The tech echo chamber really needs to get over itself.
Our customers certainly do. We get excellent results from asking a few simple questions now and then, providing both a good source actionable feedback on feature requests and any current problems, and often some encouraging comments that reassure us we are basically doing things that our customers like.
It doesn't just have to be surveys with lots of participants, though. For example, we've known for decades that a simple observational study with just a handful of people is often enough to identify most of the serious usability problems with an interface.
The idea that everything important must be reduced to automated analytics and number-crunching is a very strange disease. Even if the numbers don't lie -- and as we see here, that is far from guaranteed -- you still need to be asking the right questions and comparing useful alternatives for the results to be valuable.
"...someone recently explained to me how great it is that, instead of using data to make decisions, we use political connections, and that the idea of making decisions based on data is a myth anyway; no one does that." -- from https://danluu.com/wat/
Sorry, your comment just reminded me of that. Are surveys perfect? No, but they have their uses, and plenty of companies find real value by making use of them.
I don't use Google analytics but I have seen time and again vocal users who seem ignorant of their own usage of an application...and very much not representative of the majority.