[kqr]

I disagree. It's unfortunate that some users don't know what they want, some want things that don't exist, and that some are cats. But most users are humans with a rough idea of an existing thing they are looking for. It's worth it for a search solution to find out how good it is at helping them. The cats add noise to that measurement, they don't invalidate it.

Do you philosophically agree there are websites that are more successful than others? If yes, then there are tangible qualities that distinguish this group from the other. They may be subjective, fuzzy, and hard to pin down, but they're still there. If no, a success measure is irrelevant to you but other people might disagree, and once thoroughly investigated, you sort of have to agree the measurement coming out of it reflects their idea of success.

In none of this am I saying it's simple or easy (I started this subthread by saying it's difficult!) but fundamentally knowable.

Yes, humans talking to humans is definitely the start. But then I'm posivistically enough inclined that I think with effort we can extract theories from these human interactions.

[withinboredom]

I didn’t go into all the problems with “successful websites” but it really is impossible to measure. For me, my business site is successful when I capture leads, my blog is successful when I write posts, a restaurant is successful when people show up to eat. There’s no way of knowing what variables and metrics constitute success without asking the person.

I had a CEO who searched for the related business search terms every morning. No clicks, he just wanted to see the ranking. The other day, I was searching for an open NOC page that I knew existed but couldn’t remember the search terms. Eventually I gave up, but I’m 90% sure I left the tab open to a random promising search result that had nothing to do with what I was really searching for. There’s a pdf that archive.org fought over and simply mentioning it results in a DCMA, you can find it now, but for nearly 20 years, you could only find rumors of it on the internet and a paper copy was the only way you could read it.

Even when I know what I’m looking for exactly, I sometimes open a bunch of tabs to search results and check all of them, (This is actually the vast majority of my non-mobile searches) especially because the search results are often wrong or miss some important caveats — especially searching for error messages.

The only way you could find out these searches were unsuccessful (or successful) is to ask. There’s no magic metrics to track that will tell you whether or not my personal experience found the search successful.