[benja123]

I say this a lot and I will keep saying it. Conversion != customer obsession. There is a place for A/B testing. It is necessary and can be extremely beneficial in helping your customers enjoy and use your product more successfully.

The main issue is that people mix conversion with customer obsession! Whenever you work on a product or feature you should be asking yourself "Is this really good for my customer" - if the answer is no, then no matter what the A/B tests/conversion rates show you don't do it.

Unfortunately we mostly hire the wrong people as PMs, who then hire clones of themselves. They are not truly customer obsessed and use A/B tests incorrectly which results in products that trick or force customers to do things they don't understand/want to do. Long term this is bad for the product and company

[10x-dev]

My 'favorite' silly thing PMs do is UX research studies (typically on 5-10 people) and essentially ask completely untrained people if we should go with X/Y or Z. It's a super-effective way of avoiding responsibility for product decisions ("the data suggest we should go with Y"). If only building good products were as easy as asking what customers think they want.

[HWR_14]

Either they're doing the UX research wrong or (more likely) you're misunderstanding the process. You don't ask them if you should do X/Y/Z. You ask them to do X in the program, and see that none of them can find widget Y which controls it because they keep clicking on widget Z.

It's about observing the users fumble through your UX when you know their motivation.

[ThalesX]

> It's about observing the users fumble through your UX when you know their motivation.

Some time ago we did such a test. We called 10 customers to our offices and had them do some flows in the application. They didn't fumble. They pretty much did what they had to do and left positive reviews.

That whole thing got scrapped because consultants convinced our CEO that qualitative data is not good for global scoped startups, and that we should be building based on quantitative data.

Honestly, in less than a year, our customer experience was already taking a dive because all the extra little features we would add and strange UI elements, it became a confusing mess and our tracked NPS (Net Promoter Score) showed that. I've since left the company, but I check on them from time to time and they never really recovered and continue doing A | B in the hopes of hitting that sweet spot. It's just an unrecognizable monster at this point in my opinion.

[civilized]

Data analysis is the lowest common denominator of business thinking: the simplest, easiest thing that feels meaningful and objective. Anybody can sum up two lists of numbers in Excel and see which one is bigger.

[10x-dev]

I wish the problem were my misunderstanding the process, because then I could fix it easily by learning more about the process. I do get where you're coming from though.

[throwaway98797]

only listen to customers problems and never their solutions

[time_to_smile]

The term "customer obsession" has become a red flag for me when interviewing because I've never worked at or chatted with a company that had "customer obsession" as value that wasn't aggressively working to squeeze every dime from their users with zero interest in whether or not this squeezing was harmful to the customer.

An actual, sincere customer obsession (and btw I think we both completely agree here) means that you are willing to lose out on some conversion and revenue in order to make sure your customers are top priority.

Real customer obsession isn't just an ethical principle either, it makes business sense. The problem is that the value of customer obsession is realized over the span of years or decades. Companies that have a sincere customer obsession are the kinds of places that survive economic ups and downs, where people's children grow up and are loyal to the product because they remember the time their parents were treated well by the company.

If your only company focus is Q4 KPIs then you really can't have "customer obsession".

[jklinger410]

> The main issue is that people mix conversion with customer obsession!

The logic is: If they hate your app, they won't spend money. If they love your app, they will. Which is what would make you think A/B testing and UX work are the same thing.

There's really nothing new about this issue at all. Playing towards the average creates a lot of shitty stuff, in apps/websites as well as politics and wherever else there are metrics to track.

The genius of a good product is that it will make a stand and not give in to the whims of over-optimization in order to maintain its original intent. This is what made Apple unique.

It requires leadership with guts who aren't chasing the latest shiny object.

[jfoster]

Yeah, this is key. Improving a product in the direction of customer intent vs against customer intent.