Y
Hacker News
new
|
ask
|
show
|
jobs
by
danparsonson
1129 days ago
Yes SPAs will be next to impossible for a tool like this; I'm not sure how any tool could archive such a site tbh?
1 comments
_a9
1129 days ago
I use browsertrix-crawler[0] for crawling and it does well on JS heavy sites since it uses a real browser to request pages. Even has options to load browser profiles so you can crawl while being authenticated on sites.
[0]
https://github.com/webrecorder/browsertrix-crawler
link
[0] https://github.com/webrecorder/browsertrix-crawler