|
|
|
|
|
by paxys
1555 days ago
|
|
> The way detection works is we count the number of uBO blocked requests on the page, and if too many (threshold is set to 5), we kick it out, leaving only "clean" pages in the index. I'm genuinely surprised there were any pages left to crawl. Unfortunately this also kicks out genuinely useful blogs and other pages that are otherwise helpful but happen to be using a platform or framework that makes a few block-worthy requests. I can't figure out if all of Wikipedia is in the removed set or just ranked too low to show up in results. On the browser, the site seems clean. |
|
Turns out it was filtered out by mistake, back now!