[johnnyg]

"Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable."

Authentic question: how would a business know this otherwise in an actionable and effective way?

[peteretep]

Server-side analytics will tell you in most cases, and if they don’t, they’re easy to add in a way that makes them unlikely to be blocked?

[eli]

Server side analytics are overwhelmed by bots and crawlers and other non humans in my experience

[hinkley]

Maybe we should we whitelisting for analytics. Instead of perpetually trying to identify bots, Pick the 20% of your users who you are pretty sure are actually humans. Nielsen-style, if you will.

Maybe two cohorts. The people we're really sure about, and the ones we're pretty sure about. If they diverge, ask why.

[saagarjha]

> Instead of perpetually trying to identify bots, Pick the 20% of your users who you are pretty sure are actually humans.

Note that reCAPTCHA thinks you’re a bot if you block analytics…

[mikro2nd]

Not that that matters to me since I have (re)captcha blocked, too...

[wizzwizz4]

Blocking reCAPTCHA has lost me hours of work, repeatedly. And I still do it.

The Textarea Cache Firefox extension https://addons.mozilla.org/en-US/firefox/addon/textarea-cach... is incredibly useful, but I keep forgetting to install it when I get to a new machine. I forget only once.

[lelandbatey]

In general it's quite easy to filter out bots and crawlers from your basic access logs, as most bots and crawlers will identify themselves as such.

If you're running anything with an API, then unless somethings horribly wrong it's even easier: look at the number of requests being made to an API endpoint and spot check a few of the user identifiers (tokens, keys, whatever you're using) to see the variety of users.

All of this is assuming you're trying to merely investigate the volume of use of a feature, not trying to diagnose demographics. If you're trying to extract more fine-grained detail, I don't have as many answers; I hope others will chime in with constructive ways to get things like geographic demographics via server logs.

[eli]

A very sizable portion of bot traffic does not identify itself as such. I don’t know if it’s a majority now, but it could be.

[xapata]

Many bots and crawlers are designed to be indistinguishable from humans.

[stan_rogers]

That sounds like a server-side problem, not a client-side one. Don't expect me to solve it for you at my end.

[ocdtrekkie]

Just ask people!

[duxup]

People are bad at analyzing their own behavior.

I don't use Google analytics but I have seen time and again vocal users who seem ignorant of their own usage of an application...and very much not representative of the majority.

[yjftsjthsd-h]

People are bad at analysing metrics, too. You make a change and users spend half as much time on a page. Did you make the page twice as easy to use, or so bad that they gave up?

[saagarjha]

I honestly think that a simple “we are considering removing feature X, please let us know if you’re still using this” can work extremely well. Everyone who cares about the feature will be sure to write in.

[cortesoft]

How? In surveys? No one does those.

Plus, people often don't give valuable feedback when asked questions about features they want and use... people are poor judges of what they actually want, and will list things they think they care about and then end up never using or which don't affect their choices.

[reaperducer]

In surveys? No one does those.

We do. We have real customers do real surveys of our real web sites. In person. We even do shadowing to see how real people use our sites in their daily work.

It's uncommon in SV due to laziness and an unwillingness to talk to actual human beings. Which is dumb because there are companies that will handle this for you.

But SV is stuck in this mindset that everything can be solved by an algorithm. It can't. The tech echo chamber really needs to get over itself.

[Silhouette]

How? In surveys? No one does those.

Our customers certainly do. We get excellent results from asking a few simple questions now and then, providing both a good source actionable feedback on feature requests and any current problems, and often some encouraging comments that reassure us we are basically doing things that our customers like.

It doesn't just have to be surveys with lots of participants, though. For example, we've known for decades that a simple observational study with just a handful of people is often enough to identify most of the serious usability problems with an interface.

The idea that everything important must be reduced to automated analytics and number-crunching is a very strange disease. Even if the numbers don't lie -- and as we see here, that is far from guaranteed -- you still need to be asking the right questions and comparing useful alternatives for the results to be valuable.

[Jach]

"...someone recently explained to me how great it is that, instead of using data to make decisions, we use political connections, and that the idea of making decisions based on data is a myth anyway; no one does that." -- from https://danluu.com/wat/

Sorry, your comment just reminded me of that. Are surveys perfect? No, but they have their uses, and plenty of companies find real value by making use of them.

[briandear]

Ask thousands of people?

[reaperducer]

Or have a company do it for you. There are plenty of them out there.

[speedgoose]

You ask a sample.