I suppose a human could spend 10 seconds per Captcha, so they could do 360 per hour. Add some overhead for not being operating at peak performance every minute of every hour & call it 250. Let's say you can hire someone for $2, that works out to a bit over a penny per Captcha.
I don't think OpenAI has published pricing for GPT-4 Vision yet, but if we assume it's on par with GPT-4, and uses only 1000 of the 8000 possible tokens to process an image that's 3 cents per Captcha.
Doesn't seem completely unreasonable that at-scale humans may actually be cheaper than LLMs at this point. My mind is a little blown.
You'd be surprised, or perhaps horrified, by how cheap (self-proclaimed) human-based captcha solving services are.
If you just search for "captcha solving service" the first few results that come up offer 1000 solves of text-based captchas for <= $1 USD, (puzzle / JS browser challenge captchas are charged much higher).
Whether these are actually human based, or just impressive OCR services, it seems like they are still much more cost effective than GPT-4 is for now.
The way these work is usually presenting an existing captcha to another human who doesn’t even know they’re solving the captcha. For example, sites hosting pirated content serve fake captchas as a way to make money.
Given that it fails by hallucinating the structure of the challenge instead of refusing to solve a CAPTCHA, I doubt they've intentionally reduced the capability. Although the example in your sibling comment implies it should have enough information to do it.
