[ghshephard]

As interesting as this may seem - take it with a grain of salt. One very effective way of biasing these surveys are to run them a number of times - if there is any significant variance in the results - you can always ignore the studies that didn't work out well, and publicize the one that does.

Far better for a third-party to commission these studies and do them once, and see what the results are.

[lukevdp]

yes, I would love to see third party replication for this result.

Another thing to keep in mind with this study is it tests users in an abstracted environment. E.g., things you might find useful when comparing two sets of ten results may not be the things you find useful when you're looking for a hotel in new york.