[kuhhk]

A blog article would be nice. Sounds interesting, but I’m having a hard-time understanding. If it’s replaying requests, how do you get it to do things like go to the next pagination and click on all of the next paginated results?

[aboutruby]

In my case I can't do the pagination automatically so I have to fetch the pages myself to then have them replayed.

In most cases you would capture the request and change the "page=" parameter (either for an HTML page or an API).

You could also use selenium to click on each "next page". Could be parallelized with multiple tabs / windows.

The only website that blocks me is Bloomberg because they detect mitmproxy (I didn't care enough to make mitmproxy harder to detect).

Another detail is that regular Chrome doesn't let you load insecure certificates while chromedriver allows that.

Anyway, I will write about all that, I already posted some code on my Twitter: https://twitter.com/localhostdotdev (that I will turn into a blog).