Hacker News new | ask | show | jobs
by vie00001 1208 days ago
> I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases.

I think for search engine crawlers there are versions without a paywall so these articles can get fully indexed. Archive.ph, and similar services, might get the full content this way somehow. But I am just guessing.
3 comments
qingdao99 1208 days ago
Yes, it's called dynamic rendering when you serve different content based on the user agent, but it's apparently not recommended anymore.

https://searchengineland.com/google-no-longer-recommends-usi...

link
maxboone 1208 days ago
archive.ph also uses (donated) logins to archive (paywalled) content, however those accounts do get blocked from time to time.

https://blog.archive.today/post/678202832257794048/why-cant-...

While pretending to be GoogleBot used to get you full articles (or grabbing them from cache) this doesn't seem to be the case for some sites anymore.

They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

link
jhncls 1208 days ago
> They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

Many consumers often wouldn't read more text anyway. About one paragraph might even be too much to fill the modern attention span.

link
crakenzak 1208 days ago
you're spot on, this is exactly how sites like archive.org, archive.ph, or even if you click on "view cached version" on Google get the non-paywalled versions.
link