I agree. Its a pretty tough problem, however it is good to cross the bridge when it comes. If the search engine stays really niche, perhaps it may not even be worth it for the spammers, while doing enough to cater to the somewhat self-selecting audience. For example, the number of people who want to get to the front page of HN is likely to be a really minuscule fraction of people wanting to get to the top of search results.
Also, I wonder if is it possible to detect promotional content by analyzing things like call to actions and such?
Interesting that if this were a parsing problem (i.e. https://news.ycombinator.com/item?id=12478538), folks would immediately suggest accepting known good output, instead of trying to blacklist specific problems. The analogue in search would be something that looks more like a directory than an internet wide search engine.
Of course what killed directories in the early web is that they had no hope of scaling.
If they hide their sell links, shipping baskets, closing pages and such then they'll kill the sales though.
You can have paid news, but it's not doing anything if the mark can't buy the product afterwards because you had to remove all associations with selling to get the news to rank.
https://xkcd.com/810/
But yes, even here on HN there are problems distinguishing between legitimate articles and paid news.