PG has posted many times about improving social media discussions using an "intellectual CAPTCHA" or simple math / logic test [1][2][3][4] - "I have often sensed that a really easy math CAPTCHA would improve conversations about economic inequality.", "Make users pass a test on basic concepts like the distinction between necessary and sufficient conditions before they can tweet."
I built one using a mix of simple math, logic, reasoning questions from social media posts and retired standardized test questions. This is not a CAPTCHA in the traditional sense. The goal is not to stop AI, but to allow users to participate in discussions.
- Invites are temporarily open to HN users.
- Onboarding test + one daily question before accessing feed, post or reply.
I got 5/7 this time around. Top 5%. At least one of the two that I missed I think are really very difficult in that I believe they use words in strange ways.
Questions are better for sure. The maths ones were all vastly easier than the "disprove conclusion" ones, primarily because it's not always clear what the conclusion to be disproven even is (are we supposed to be doubting one person's claim, another claim, or the claim at the end?)
I expected a mix of false fact, so people has to reason from the facts in the screen and not from the their real world opinion. I'm not sure if that will be harder for humans or for AI.
Questions are better for sure. The maths ones were all vastly easier than the "disprove conclusion" ones, primarily because it's not always clear what the conclusion to be disproven even is (are we supposed to be doubting one person's claim, another claim, or the claim at the end?)