[omneity]

That's super cool, thank you for sharing! It's based on playwright though right? Can you verify if the approach you are using is also subject to the bug in TFA?

My original point was not necessarily about bypassing anti-bot protections, and rather to offer a different branch of browser automation independent of incumbent solutions such as Puppeteer, Selenium and others, which we believe are not made for this purpose, and has many limitations as TFA mentions, requiring way too many workarounds as your solution illustrates.

[erekp]

we fix leaks and bugs of automation frameworks, so we don't have that problem. The approach of using the user's browser, like yours, is that you will burn the user's fingerprint depending on scale.

[omneity]

Thanks for sharing your experience! I'm quite opinionated on this topic so buckle up :D

We avoided the fork & patch route because it's both labor intensive for a limited return on investment, and a game of catching up. Updating the forked framework is challenging on its own right, let alone porting existing customer payloads to newer versions, locking you de-facto to older versions. I did maintain a custom fork at a previous workplace that was similar in scope to Browserless[0] and I can tell you it was a real pain.

Developing your own framework (besides satisfying the obvious NIH itch) allows you to precisely control your exposure (reduce the attack surface) from a security perspective, and protects your customers from upstream decisions such as deprecations or major changes that might not be aligned with your customer requirements. I also have enough experience in this space to know exactly what we need to implement and the capabilities we want to enable. No bloat (yet)

> you will burn the user's fingerprint depending on scale

It's relative to your activity. See my other comment about scale and use cases, for personal device usage this is not an issue in practice, and users can automate several websites[1] using their personal agents without worrying about this. For more involved scenarios we have appropriate strategies that avoid this issue.

> we fix leaks and bugs of automation frameworks

Sounds interesting! I'd love to read a write up, or PRs if you have contributed something upstream.

0: https://www.browserless.io/

1: https://herd.garden/trails

[erekp]

sounds good. As you can probably imagine, I also come from a lot of experience in the space :) But fair enough, everyone has their own opinion on what is more or less painful to implement and maintain and the associated pros and cons. We're tailored to very specific use cases that require scale and speed, so the route we took makes the most sense. I can't obviously share details of our implementation as it'd expose our evasions. And this is the exact problem of open source alternatives like camoufox and the now defunct puppeteer-stealth.