Ask HN: What if AI was used to verify users reading articles before discussion?

[morceauxdebois]

I recently read a frustrating thread on HN where it was obvious several people were making arguments against points that were already addressed in the article. The discussion in the thread was cluttered with people attacking straw men instead of having any fruitful discourse.

I then came upon the idea that perhaps something like GPT-3 could be used to have users answer 3 questions about the article's content before being able to comment.

[jstx1]

- The contents of the article are less important than you're implying, most people are here for the discussion. Yes, sometimes not reading the article is a problem but many other times it isn't and this can be clarified in the course of the discussion. Very often the article itself is just a starting point to discuss a broader topic.

- People won't answer the questions like you suggest, they'll just leave. In a hypothetical world where your feature is implemented, it will destroy the site instantly.

- You're making very generous assumptions about what language models can do.

- HN is a website with very minimal features. We don't even have dark mode. I wouldn't expect complex AI features any time soon. Or ever.

- Seriously, take a second to think about the user experience of a discussion site that asks you to take a test before writing a comment.

[krapp]

When people don't read the article, they tend to do one of two things - comment on the title (which is often misleading) or use the title as a springboard for politically-driven polemic. Both drive down the quality of discussion considerably. The fewer people who read the article there are, the less substantive the discussion becomes, less insightful, less novel, more tedious and repetitive. The best comments are, invariably, informed by the content of a well-written, insightful article.

The goal of this community is to gratify intellectual curiosity. If most people the forum and never engage with the posted content, preferring only to engage with other, similarly unenlightened commenters, how is that curiosity ever going to be gratified?

It's as if Hacker News were a book club and you said it wasn't that important to actually read the books. Yes, it's possible to discuss A Confederacy of Dunces by only having read the cover and back blurb and hoping someone else in the group actually did the legwork, but why join a book club in the first place if you care so little about reading?

Unfortunately, I think your other points are correct. It isn't possible or even feasible to force people to read the articles. More people have to want to put in the effort and if they don't, they don't. There should at least be as much social pressure within the community to expect people to engage with content as there is to suppress humor or incivility, but the insularity and elitism of the culture here makes that infeasible. There's no way to engineer this, people have to care more.

[brudgers]

Because HN allows comments to wander...or jump on an airplane to a different continent...reading the article is not a prerequisite for good comments and discussion. The articles are mainly an excuse to type into little boxes.

The main thing that produces good discussion is good comments. The double converse is also the case. And a recognized problem: people read the article to find something negative to say...cherry picked quotations taken uncharitably is a proven method with the bonus of suggesting that the commentor actually read the article (but it is easier if something early in the article is a call to angry typing in outrage).

To put it another way, HN is able to have interesting conversations about lists, diagrams, and landing pages. Thin low information blog posts that are only worth reading for their headlines can spark deep informative comment pages.

To put it another way (again), Ask HN is a ready example of how little catalyst the community needs at times. The article is a spark and if it lands on the right intellectual tinder you get a great inferno.

[For polemic reading the article matters not. Polemitians bring their own tinder to the forest and find fuel in the comments no matter how good.]

[tomjen3]

Old angry man rant incoming.

>comment on the title (which is often misleading)

I grew up with the understanding that in a newspaper you should be able to read the title and get to know what happen and then be able to progressively read more if you wanted more details.

When the title is not accurate enough, or is click-bait, maybe we should start there with the AI there?

[jstx1]

Newspapers use (used?) an inverted pyramid structure because the bottom of the article can often be cut in editing so everything fits correctly on the page. As a newspaper journalist, this is out of your control so you need to make sure that your text makes sense even if one or more paragraphs are cut from the bottom.

I suppose it helps with attention and assimilating information too.

[onion2k]

If I found my posts were blocked because some AI said I wasn't posting correctly I'd leave and never come back[1]. The chilling effect this would have on discussions here would have a much more detrimental impact than any benefit from preventing bad posts.

[1] Some people might argue this is a benefit.

[morceauxdebois]

Your posts wouldnt be blocked, you would have to verify that you have read the article first before posting.

[onion2k]

By that you mean you'd have to have read the article and interpreted in the manner by which the HN AI has accepted, right?

HN is often accused of being a 'hive mind' of users with homogenous opinions already. Making people assert that they've read the article, and understood it within a set of constraints, could remove a lot of nuance and variation.

Plus, I hardly ever read the articles.

[yaitsyaboi]

Most articles have facts that don’t require (very much) interpretation.

Provide the name of a city mentioned in this article.

[nathanaldensr]

Sounds like barely more than a captcha.

[brudgers]

How do I prove I read a repository or a landing page?

[bjourne]

I have noticed that too and it is pretty fucking annoying (+). If the article is about US colleges then the first comment is always about over education and how not every job requires a college degree. If the article is about trains it is about how the US, unlike Europe, is too large for train networks. If it is about renewable energy, the first comment is about how the green movement is shooting itself in the foot by shunning nuclear. And so on.

I think you could at least partially solve the problem by having the ranking algorithm bias more heavily towards recent comments. As it is now the first comments are much more likely to reach high scores than later comments, which is stupid because it might take an hour or two to read an in depth article and formulate something insightful.

+ - And I've been guilty of it too.

[chroem-]

Many people would already consider HN to be a bay area-centric echo chamber. Based on the examples you listed, this seems to be an argument to explicitly enforce that echo chamber so that nonconforming posts are blocked outright. At that point, why even bother allowing users to discuss things at all, if posters are only allowed to agree with each other?

[jgwil2]

Nowhere does GP suggest anything of the sort. Only suggestion is to weigh upvotes less heavily relative to recency in determining comment order.

[trompetenaccoun]

People making fallacious or otherwise nonsensical comments is a problem all over social media. Like you say it's common that articles aren't properly read, and I'm sure most of us are guilty of it sometimes. There can be many other possible explanations though. For one, there are and always have been trolls. We're all different ages, education levels and intelligence here as well, that can make a big difference because some simply don't understand why their arguments aren't logical, no ill intentions. Lastly we should consider that GPT-3 & co are already here. Probably less on HN than Reddit, Fakebook and Twitter, but it's obvious that governments and corporations are using bots to influence social media in general.

So how are you going to protect against this? ANNs are great at answering these questions, and the AI can't outsmart itself. By making it harder to comment you'd actually lose more human commenters, because like others say what you suggest is frustrating, most people wouldn't bother. We'd be left worse off.

Imo rather than putting computers in charge, we might soon have to consider the opposite: Checks for users to verify that they're human. This will probably have to be done by humans themselves, but I've not seen a really practical idea of how it could be done yet.

[muzani]

I don't think this would solve it. AI can make mistakes too and quite frequently. As we've learned from CAPTCHAs, they're very annoying and kill engagement.

I think the moderation and "the post answers this" works well enough. If there are lots of straw men, it could be a problem with the content or a clickbaity title.

In many cases, I don't care much about the article at all. Sometimes it's just "new update from Firefox" and people just want to talk about Firefox updates.

[antisthenes]

> I recently read a frustrating thread on HN where it was obvious several people were making arguments against points that were already addressed in the article.

Ummm, yes? Isn't that the point of a discussion thread? Just because the article makes some points doesn't exempt them from criticism.

Many articles posted on HN are outright garbage - e.g. very poorly constructed studies or some undergrad psych/nutrition juju that doesn't pass a basic critical thinking test, so it's quite rightful that people don't waste their time reading them (or at least reading them completely)

> The discussion in the thread was cluttered with people attacking straw men instead of having any fruitful discourse.

Typically, people attacking straw men will do so regardless of whether they read the article. The problem with those people is that they invent the straw man, attack it, win, and do all those things without ever caring if it relates to the topic at hand at all.

[ynac]

I like the idea of knowing who has read an article in full and who has not.

That said, I don't mind some noise from people who scanned (or less) the article. Random and wrong is still fuel for my thinking. Which is why I'm here. I don't have intimate connection with individuals here so much as the group as a whole.

To respond directly to your suggestion, I'd like ANY (even honor system) notation for people who have read the article (RL1, Reading Level: 0-3).

I like the Q and A idea for myself so I can review the article mentally and recapture elements and think more deeply on the subject for retention AND for better participation in the conversation. I would actually like that on my browser - "quiz me on this page" - anyone want to collaborate? Not that I'm a chatty Charlie on the boards here, but it is nice to be more confident in one's ideas before sharing.

[grawprog]

>The discussion in the thread was cluttered with people attacking straw men instead of having any fruitful discourse.

Isn't this partly why the downvote button exists and pushes lower voted comments down and greys them out or the flag button for those really terrible off topic comments?

[eivarv]

NRKbeta, the tech vertical of the Norwegian public broadcaster NRK, actually did the manual version of this a few years back [0-1], but found it hard to conclude with success or failure based on the evidence they gathered.

[0]: https://nrkbeta.no/2017/08/10/with-a-quiz-to-comment-readers...

[1]: https://www.niemanlab.org/2017/03/this-site-is-taking-the-ed...

[version_five]

I find HN has a better SNR than any other user discussion I've read on the internet. Arguing against points addressed in the article seems like a good thing to me (maybe I misunderstand though and you mean people are missing objective facts). My biggest criticism of the discussions is that there usually isnt much diversity of opinion for controversial topics and most outside a narrow range get downvoted. I think diversity would get worse if there were more barriers to commenting.

[jgwil2]

I think the complaint is that people routinely rehash points that were already addressed in the article, because they didn't bother reading it in the first place.

[GOATS-]

> I then came upon the idea that perhaps something like GPT-3 could be used to have users answer 3 questions about the article's content before being able to comment.

The Norwegian equivalent of the BBC did this.

https://arstechnica.com/gaming/2018/03/how-a-norwegian-comme...

[saternius]

You can check out our best attempt at a question generator at http://quickquiz.me, to get an idea of what type of questions a large language model (T5) can produce. Its the best model my team and I witnessed. We are still collecting data to generate more salient, and well structured questions. Its a work in progress.

[sircastor]

This reminds me of a handful of courses I had in school where we had to read an article, take a quiz and then participate in a “discussion”.

To me, the “you didn’t read the article” or “the article directly addresses this” responses are most effective dismissal of such a post, and a bit of a public embarrassment for the poster.

[sobellian]

GPT-3 is neat, but I definitely get "2+2=5" vibes when reading some of its output. There's a lot of content that's statistically plausible but not logically rigorous. The last thing I want is for this species of algorithm to then screen what I post for intellectual rigor.

[Leparamour]

The marketers love your idea and will inject even more ads if reading the article until the very end (no matter how devoid of information or misleading they may be) becomes mandatory just to comment on it.

[mikewarot]

It feels like half of the time, you can't read the article because of a paywall. Usually somewhere down the comments someone will have posted a way around the wall, which helps.

I share the view of others here, the article is just a seed crystal for discussion. Sometimes the most valuable comments are found in the misunderstanding and tangents that arise.

[tomjen3]

I don't read the article and basically never without first reading the comments. Far too many times I see somebody here who can disprove, or knows more about the topic than the article.

[eimrine]

Gpt-3 can not make questions from text. If would be better to give instant answers if it can read and realize the material, because sometimes I really do not need to read all the article.

[muzani]

There's a discussion on how to do it here: https://community.openai.com/t/ask-and-answer-questions-from...

This was with half an hour of work. I'm sure that with proper R&D, it can do better.

[aboringusername]

"Sorry due to GDPR you cannot view this article."

"Hi - here's 50,000 ads and 1PB of JS loading in your browser and the content jumps around, have fun! Oh btw pls accept cookies from 1,000,000,000 ad networks, kthx".

Most websites consume ungodly amounts of JS, are janky as shit and deplete battery life and consume resources pointlessly and unethically.

If an article loaded as fast as HN does then sure, after all, it's just words on a page like this thread, not hard to render in 100kb of data transfer?

Sadly, because the cancer of ads has infected the internet and we treat it less severely than Covid we get a situation where reading an article becomes choresome.

Fix that then maybe you have a chance but that's why I rarely read the article unless it works on noscript and can load in dial-up style internet conditions. You've got around 3-4 seconds or I'm out.

[Graffur]

You could do it as a Firefox extension which then gives the users who answered correctly a green tick or some other indicator.

[slightwinder]

Then people would build a solution to automatically solve the verification-process...

[l0b0]

(Go ahead and downvote, but I think the tone in this answer is appropriate.)

Yes, let's sprinkle magic fairy dust on threads to fix human foibles. Let's make sure there are no more meta-discussions, like people not being able to read the article for any reason. Let's also avoid any posts like "Hey, if you like the author you should check out X". Or, conversely, "Hey, this person is a known crank[1][2][3], don't bother reading." Or anything where a one-paragraph tl;dr could've easily replaced the entire article, and answering based on the tl;dr is completely reasonable. Or any of a host of other types of comments which I couldn't think of off the top of my head.

Let's instead make sure everyone who wants to post needs to remember things from the article which are completely irrelevant to the actual content, and that they spend a few minutes trying to guess what the question means and which exact characters will appease the algorithm.

Let's not.

[paulcole]

> I recently read a frustrating thread on HN where it was obvious several people were making arguments against points that were already addressed in the article.

HN wants the engagement and doesn’t give a hoot if people read the articles before commenting.