[colordrops]

It would know more than that you are interested in headphones. A particular URL that you could upvote or downvote has a ton more information than that - the website itself, the part of the website it is on, the content of the page, other websites it links to. Legitimate sites tend to link with other legitimate sites.

[Gigachad]

This is all stuff Google already does. Rather than requiring specific upvotes they can generally tell how satisfied with the result you were if you drop off rather than coming back to click more links. Problem is all of these signals are gamed to the extreme.

[colordrops]

If that is true then whey do they always suggest Pinterest and Quora at the top when I've been ignoring them for years? If there was a downvote button they would be gone from my results.