[x1xx]

> Spotify saw a 9% increase in exploratory intent queries, a 30% rise in maximum query length per user, and a 10% increase in average query length—this suggests the query recommendation updates helped users express more complex intents

To me it's not clear that it should be interpreted as an improvement: what I read in this summary is that users had to search more and to enter longer queries to get to what they needed.

[rorytbyrne]

We would need to normalise query length by the success rate to draw any informative conclusions here. The rate of immediate follow-up queries could be a decent proxy for this.

[Traubenfuchs]

> a 9% increase in exploratory intent queries

Users struggle to find the right stuff or stuff that‘s so good they don‘t need do do more queries.

> a 30% rise in maximum query length per user, and a 10% increase in average query length

Users need to execute more complex queries to find what they are looking for.

[RicoElectrico]

I can understand tracking metrics for performance (as in speed, server load) or revenue. But I don't see how anyone could make such conclusions as they did with a straight face, apart from achieving some OKR for promotion reasons. There's no substitute for user research, focused mindset and good taste.

I can imagine that's why today's apps suck so much as most of the pain points won't be easily caught by user behavior metrics.

One thing Alex from Organic Maps taught me is how important it is to just listen to your users. Many of the UX improvements were driven by addressing complaints from e-mail feedback.

[RamblingCTO]

100%. I've switched over to Apple Music because you really feel that they are pushing public playlists. Search terms maximize their playlists vs mine. I now had to go to my library to find my playlists because they wouldn't even show up.

[singron]

This is a hard problem. We had similar issues evaluating success with real users. In the literature, there is "abandonment" (i.e. I couldn't find what I wanted and gave up) and "positive abandonment" (I got what I wanted from the SERP and didn't click on anything). A flurry of requests might be a series of positive abandonment, a natural fruitful process of refining the request, or rage querying where the user repeatedly fails to correct a model that is incapable of understanding the query. It's especially devious if they rage query for a while before switching to an easier task and succeeding (e.g. clicking a result) since you might count that whole interaction as positive when it was really quite negative.

[barrenko]

People are just more and more used to interacting with an LLM / GPT, I think that's the why of the long questions + yes, people are not finding what they need.

[1oooqooq]

that's what you get when you have a "search pm".

[braiamp]

Yeah, this should be evaluated in a multivariate/bivariate model. Of the successful queries, how the length changed before and after interventions.

[wildrhythms]

No you don't understand, more queries = more engagement!

[MostlyStable]

It's relatively easy to construct a scenario where more search is in fact indicative of better search. To stick with Spotify: let's imagine they have an amazing search tool that consistently finds new, interesting music that the user genuinely likes. I can imagine that in that situation, users are going to search more, because doing so consistently gets them new, enjoyable music.

But the opposite is equally possible: a terrible search tool could regularly fail to find what the user is looking for or produce music that they enjoy. In this situation, I can also imagine users searching more, because it takes more search effort to find something they like.

They key is why are users searching. In Spotify's case I imagine that you could try and connect number of searches per listen, or how often a search results in a listen and how often those listens result in a positive rating. There are probably more options, but there needs to be some way of connecting the amount of search with how the user feels about those search results.

And yeah, using nothing other than search volume is probably a bad way to go about it

[cco]

Or more saves and thumbs up on signs resulting from a search is because users are desperate to save a song they like because they have no faith that they'll be able to find it again with search.

The only way is to use the product yourself and honestly engage with it. Stats can't answer this question.

[genewitch]

Contextually unique searches versus contextually similar searches.

Nin, NIN, nine inch nails, Trent Reznor

VS

Nin, pantera, nail bomb, muse

This should be easy to differentiate, with a "[someone's name] distance algorithm" or such, right?

[jimmyl02]

I feel like understanding this difference is what a good product manager should be responsible for. Not just optimizing any metric that is available but understanding the meaning behind them and choosing the push them the right direction.

[whatevertrevor]

But isn't that actually the point? That measuring query volume tells you nothing?

[MostlyStable]

Yes, I was agreeing and expanding on the point.

[porridgeraisin]

A conclusion true to the concerned executive's MBA.