I think GPT-3 could be blessing for Reddit and HN type forums.
Add some GPT-3 content and links into the feed. Decrease the vote weight for users who upvote them and increase vote weight for those who downvote them.
I've noticed a trend recently in a subreddit I frequent that obviously wrong statements are suddenly getting upvoted. I'm not talking about things that conflict with my opinions - I'm talking about statements that are demonstrably and objectively incorrect.
I think the issue is a critical mass of people very new to the topic at hand who upvote things that sound reasonable without having the knowledge to be able to engage with the statements even slightly critically. This forms a feedback loop as upvoted comments are assumed to be reasonable. It means you can say something blatantly wrong, but stated in a manner such that it assumes the form of a sensible insight, and be upvoted for it.
Something like the system you propose would do well to promote genuine insight above platitudes, and platitudes above superficially plausible misinformation.
I don't see this happen often in areas where I'm an expert but I do see it, and it's weird.
There's a post somewhere, maybe one of the Stack Overflow sites or Reddit where somebody asks about how SSH works, and the answer given and upvoted is horribly wrong, it's like how somebody who half-understood an explanation of PGP might think SSH could work.
So I down-voted that and I wrote an explanation based on my understanding but referring to the RFC as I went, and, whenever I was surprised by the RFC, also checking the OpenSSH source code (the RFC is correct, but, you know, always worth checking).
It got downvoted. Zero comments. So clearly people are looking at these two explanations that are quite different and they are down-voting the one they... don't like?
Or even better, for repetitively-discussed topics, just generate the entire comment thread to save us all time. Examples: nuclear power, housing/transit policy, Tesla/Elon Musk.
Any Show HN gets a top comment saying it's a nice pet project but pretty useless seeing as there's some other tool out there that does the same thing except better
I agree with everyone: this is an intriguing idea.
I feel like if we did it, it would be sporting to reveal which comments they were, after a suitable amount of time.
One tricky bit is which accounts would post the comments. If they were all new accounts that hadn't posted anything before, that would lessen the value of the test. I'm not sure it would make sense to have dedicated accounts for this. Perhaps we'd have to sprinkle such comments among established accounts? With permission from the account holder, of course, plus swearing them to secrecy? That starts to sound complicated.
(The thread at https://news.ycombinator.com/item?id=24006393 shows that GPT-3 can already pass my screening turing hurdle, one which many actual people arguing on the internet fail.)
Two academics, Albert and Bill, are sitting in a bar waiting for their friend Charlie.
Albert: "Charlie thinks women don't know any math, he might be right, but I want to play a trick." Albert calls the waitress over.
Albert: "Delia, my friend Charlie is going to arrive in a bit, and I want to play a trick. When he arrives, I'm going to ask you a question, and I want you to answer, 'X cubed divided by three.' Can you remember that?"
Delia: "Sure, no problem, I can remember that."
Charlie arrives and Albert raises his favorite topic.
Albert: "I think you're wrong, I think women can learn math. Just as a test, let's ask the waitress a math question." Albert calls Delia over.
Albert: "What's the integral of x squared, derived with respect to x?"
Delia: "Umm, that would be ... x cubed divided by three"
I'd like to see a more dynamic interpretation of this.
1) A form has seed users, extrapolate how they would vote based on people who vote like them but see more content. Use the extrapolated prediction of seed behavior to rank.
2) Let the viewer change who their seeds are manually. Let the viewer rank the posts, and see which seeds work best for them. Make this process and equilibrium building dynamic.
You can avoid ethical concerns by ensuring that users are technically informed this might be happening. In fact you might get great results by informing users this might be happening even if it you rarely or never actually do it.
You can also phrase it quite broadly so as to ensure that participants were technically informed of why it's being done and yet nonetheless unaware of what exactly you're actually doing. That's how psychologists design experiments that want to measure something subjects would prefer to conceal because of low social desirability.
For example suppose we're wondering if people are secretly biased against rectangular shapes in video games but are feeling a social pressure not to admit this bias. We tell subjects we want to test for bias against rectangles, they're going to play a video game, they are to collide with the red objects (regardless of shape) and avoid blue objects, we will show how many rectangles they hit on the screen.
But we don't actually care about the count, we actually use eye-tracking technology to measure which objects on the screen the subjects look at, when and for how long and we use this fact, that the subjects don't realise we care about, to check for bias.
Isn't every single ad ever created experimenting on users without their knowledge? I.e., will the user click on this ad or buy the product if we show them this content? That has much greater probability of adverse consequences for the user than the proposed experiment.
I've noticed a trend recently in a subreddit I frequent that obviously wrong statements are suddenly getting upvoted. I'm not talking about things that conflict with my opinions - I'm talking about statements that are demonstrably and objectively incorrect.
I think the issue is a critical mass of people very new to the topic at hand who upvote things that sound reasonable without having the knowledge to be able to engage with the statements even slightly critically. This forms a feedback loop as upvoted comments are assumed to be reasonable. It means you can say something blatantly wrong, but stated in a manner such that it assumes the form of a sensible insight, and be upvoted for it.
Something like the system you propose would do well to promote genuine insight above platitudes, and platitudes above superficially plausible misinformation.