[mdaniel]

Polling is a fool's errand, in my experience, since I'd guess a ton of the "how about now?" responses are "nope"

IMHO sitemap.xml and/or ActivityPub are sleeper approaches to helping the sites with the adversarial relationship they have with scrapers. In my experience, there are two schools of thought: play the very expensive game of cat and mouse (expensive for BOTH sides) of trying to verify the human-ness of visitors, or make it so the inevitable bots don't eat up very expensive and valuable resources that can be spent serving actual humans

Imagine how much less nonsense would have to go on if Amazon had ActivityPub that CamelCamelCamel could subscribe to, or similar for Craigslist

I'm not saying "all information wants to be free," so if Amazon wants to focus its energies on hoarding the review content, since that's arguably its moat, fantastic - let the bot and anti-bot people war over that content, instead of arguably factual data found on the product listing pages

[martyvis]

Maybe there is a language issue here. In Australia, a full on election is also called a poll. The government electoral commission does the polling, the voters do the voting.

I think you getting confused with opinion polling.

[mdaniel]

Ah, that's quite possible - since it was a thread about APIs, I interpreted the question as "a way to programmatically await data arrival" - the verb "polling" in API context is repeatedly invoking a status endpoint to ask if new information is available either in absolute terms or in terms of since the last request

I stand by my rant :-D but obviously stand corrected on the question being asked