[ekianjo]

Talking to people is not going to help you either. You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions. Classical trap of market research.

[tlarkworthy]

ycombinator startup school disagrees and says it's one of the two CRITICAL things founders must have a hand in.

Of course you need to interpret it but its incredibly important and I do not think you really know what you are talking about.

https://www.ycombinator.com/library/6g-how-to-talk-to-users

Almost all the major fails I have seen in my career have been some derivative of not understanding your users.

[sopooneo]

At the very least, I feel talking to users will give you decent hypothesis to test.

The creation of hypothesis is often glossed over as a trivial first step in scientific or data-driven decision making, but in fact, that's where the magic lives.

[ekianjo]

> I do not think you really know what you are talking about.

Nice strawman. I have never said to never talk to your users, but to pretend that using data is meaningless and you should follow some bullshit and vague "good argument" instead is just sheer foolishness.

[btilly]

That depends on how big the differences you're looking for are.

When you've got an early product, there are probably things you can do that 2x as many people will like as dislike. Even a small set of customers will be good for discovering this. When you've got a mature product, you should be optimizing around the edges and need a large sample size to find those 1% wins.

Likewise if you don't have scale, there are a lot of well-known best practices that probably improve your site by 5-10%. You probably don't have sufficient volume to discover test those ideas, so following general best practices is a good idea. But if you have scale, you can and should A/B test the heck out of everything. And then do it again in a couple of years in case the answer changes.

[tlarkworthy]

It's this data-led uncritical thinking that destroyed facebook

[bogdanoff_2]

Talking to customers might uncover some things you haven't even thought about.

[ekianjo]

The same thing is true with pure data analysis. Unless you have never analyzed data in depth in your life, that should be pretty obvious.

[jzb]

You have to do both. You can't just look at data & you can't just talk to users / customers without looking back at data.

[ekianjo]

of course, but then following "good argument" is just ignoring data in the original article, which is nonsense.

[indigochill]

> You end up getting a lot of noise and making sense of what you hear is difficult. When you keep probing you will get to hear stuff thats not really critical and just often made up because you ask too many questions.

Yeah, but this just means qualitative data is challenging, not that it's useless. You have to be careful when asking questions that you're asking useful questions and not leading people into telling you what they think you want to hear (or going off on useless rabbit trails like what they think the product should be instead of what the problem they want the product to solve is).

[czbond]

While I agree with your suggested outcome for some or many, a product designer or manager who is skilled at asking questions, going deeper, removing distractions, asking why continuously, and empathizing while not seeming judgey can garner really good insights.

I am guessing it's like you see of a psychologist with a patient on TV..... the customer must feel comfortable enough to open up, then flood gates can open.

[spaniard89277]

Go talk with your customer service. Oops, so much rotation nobody cares, everyone is cheating KPIs.

[peteradio]

You both make good arguments, there must be a middle here. I doubt you can uncover what your customer wants very well without just talking to them, but maybe they wind up misleading you sometimes. A/B testing to discover a customer wants a whole different paradigm isn't possible.

[Cthulhu_]

This is true; what customers SAY they want doesn't necessarily corellate with what they will actually use or pay for.

I mean I worked on an app where in one part, the end user could upload CSV files to be used. What they SAID they wanted was basically a full data management system and RESTful API to enforce constraints, data validation, record retrieval and updating, etc. What they probably wanted was an excel sheet. I dislike how my employer was like "yeah sure if you pay for it" to them.

[dspillett]

> what customers SAY they want doesn't necessarily corellate with what they will actually use

A key cause of this in many cases is that the stake-holders you talk to do not work closely with the end users of the system. Talking to the right people can help a lot, though unfortunately as a 3rd party this is not usually anywhere near your realm of control.

The other issue is them knowing what they have and wish to store, but not knowing what outputs are going to be needed down the line. That is harder to fix, but having some good industry knowledge within your company can be a great help on such matters – you can then sometimes preempt client needs if the people holding that knowledge are keeping an active eye on changes (for instance new/planned regulations that might be coming into force in X weeks/months/years).

[cbtacy]

Very dated thinking. Suggest you read up on Lean Customer Development (for example).

[ekianjo]

been doing product design most of my life in top corporations, so I'll pass on your opinion.