[ronjouch]

> “Edit: maybe you are thinking about the AI deal which is exclusive to Google. That's not the same thing as search engine indexing.”

@mastazi that’s what I’m talking about, and I think your AI vs. indexing nuance is incorrect. I wasn’t sure, so I just did a quick N=1 verification: searching for the name of a random 1week-old popular Reddit post with a precise unique title,

- Insta-found it on Goog as top result

- Didn't find it on DDG, with or without site:reddit.com

Looks like sibling comment from @cpressland (thanks!) is correct: as of today and until other search engines sign licensing agreements with Reddit, “non-Google search engines cannot get new results from Reddit”. See https://www.reddit.com/robots.txt , which links to https://support.reddithelp.com/hc/en-us/articles/26410290525... , section “Reddit may license public content for commercial or non-commercial use”

[mastazi]

you are right, and I was wrong;

I found this article by Ars Technica that highlights the consequences of the deal for search engines:

https://arstechnica.com/gadgets/2024/07/non-google-search-en...

so Reddit, the website co created by Aaron Swartz, is now a walled garden.

Wow.

[esperent]

I thought that DDG and Kagi get a large amount of their results from Google? Do these have reddit results stripped out?

> a random 1week-old popular Reddit post with a precise unique title

While a week is a long time, is it possible that the other search engines just hadn't got around to indexing it yet?

[ronjouch]

DDG gets a large amount of their results from Bing: https://duckduckgo.com/duckduckgo-help-pages/results/sources...

Kagi I don’t know much about, but quickly reading about it, seems to indeed pay Goog.

> “While a week is a long time, is it possible that the other search engines just hadn't got around to indexing it yet?”

I doubt it, search engines these days are much faster, and this is double-confirmed by 1. articles linked in sibling comments, 2. reading https://www.reddit.com/robots.txt