Hacker News new | ask | show | jobs
by dylan604 590 days ago
"We can't make a map of all of Twitter, because the data isn't available and scraping it would be difficult and illegal."

What makes scraping illegal?

4 comments

The LinkedIn vs HiQ case makes it so that scraping data behind a login screen is not okay but publicly accessible pages are fair game.
Was this the main reasoning for moving everything behind login more than data hoarding of the viewers?
My personal belief is yes. Legal talks to management after the case concludes, tells them this is the new reality, and management tells product to make the change.

At no time is management obligated or even remotely motivated to tell end users the real reason for making the change, because users believing that the platform is just a greedy corporation is better than them realizing that the platform will just roll over if the police presents them with a subpoena to snitch on a user.

Post-2024, with Elon running the government as fiscal/regulatory czar, Twitter policy is evidently stronger than judicial interpretation of Constitutional rights.
What happens to Twitter when Musk has to step down from his roles with Twitter/SpaceX/Tesla? Will sanity be allowed to fill the void? I really doubt anything at SpaceX/Tesla changes, but Twitter has the most wiggle room under someone else's leadership
Why would he step down? He doesn't seem the type to give up power just because something shiny is taking his attention. It's not like the incoming administration has signed a pledge to avoid conflicts of interest or anything
Perhaps even the threat of being sued might be enough to even consider doing this.
Nothing. A precedent determining that scraping is legal is already established.
"legal" doesn't imply that Twitter/X won't send a C&D due to scraping that is expensive for a normal person to fight. Moreso if Twitter/X invokes the CFAA which has been common for scraping cases post HiQ vs. LinkedIn.
not if its behind a login wall.