Hacker News new | ask | show | jobs
by staticshock 4038 days ago
From personal experience, I've tried scraping my own TD Ameritrade data and quickly got flagged by their system as a virus because my request pattern likely failed some heuristic check. They disabled my account "until I get that cleaned up" and I had to plead with multiple people over the phone just to get it re-enabled.

The overall experience was quite terrible, but I wouldn't be surprised if it's the norm.
3 comments
olegus8 4038 days ago
Certainly can imagine this happens. Currently I'm scraping 4 banks and 4 shops on a daily basis, and the only issue was when scraping for the first time from new IP address. They asked a security question or displayed a captcha. I resolved this by logging in first time from the browser through SSH tunnel over that IP. All subsequent scraping went well since then.
link
stavrianos 4038 days ago
Opposing personal experience - I've been scraping my bank data (scotia) for about a year now, haven't had any serious problems (there seems to be some kind of signin counter that thresholds and then forces secret questions to be reset, but that's it).
link
walterbell 4038 days ago
A concerted community effort could develop counter-heuristics.
link