Isn't the whole reason behind the 'points' part of the HN comments to hide inane comments? If a comment is inane, it will be down-voted and grayed out. I think relying on humans is a far better filter for inane comments than number of syllables in a sentence.
That said, the code looks good and the idea is fun - might be nice to try and implement a javascript NLP library to make it more intelligent? [1]
This algorithm seems incredibly arbitrary and fails to account for the complexity of diction or sophistication of content of the post. (Obviously, this is incredibly difficult and computationally-intensive, and is likely untenable for a simple Chrome app, but applying such a narrow and draconian algorithm that reduces such complex facets of pedagogy and language into a calculus comprised only of sentence size and syllabic length does not help in weeding HN of poor-quality content.)
Moreover, I'm not sure as to why I would want to rid HN of "8th Grade Content" in the first place. I'm fine with terse sentences and pedestrian word choice so long as the actual content of the post is of value to the conversation.
Moreover, due to algorithmic simplicity, the results are simply wrong.
Almost assuredly, the preceding sentence could be comprehended by individuals with less education than a year of college education, yet most eighth graders would not be able to comprehend statements like "The impetus for men to exalt banal works of prose is arcane."
I'm sorry, but this assemblage is aspiring to a sophisticated strata of dialogue. Kindly elevate your contribution beyond the 2nd grade level. (http://sarahktyler.com/code/sample.php)
...alright, even my tongue-in-cheek comment only hit the the 7th grade. Methinks the true goal of "a Chrome extension which hides all HackerNews comments that are below an 8th grade reading level" is to get everyone off the comments and back to work.
If you want to filter by grade level, why not just use a ready-made javascript script that computes an actual Flesch-Kincaid reading level [1]? This seems like reinventing the wheel, though I understand that can be the point of a weekend project.
Maybe because the Flesch-Kincaid reading level isn't a particularly good metric of anything other than sentence length and syllable count? Certainly not of "content" or "value" however you might define those concepts.
Tell that to the author maybe? The author implemented the Flesch-Kincaid grade level algorithm in his plugin.
In other words, I'm not suggesting that he use Flesch-Kincaid; he already made that choice. I'm suggesting a way for him to not re-implement an algorithm that's already implemented in his language of choice.
Something to turn this into a killer extension would be "Hide comments that use words on my filter list".
Thus, I could chose to hide any comment that used the word FANBOI or fanboy or whatever.
You'd need to include some method for alerting me that there's a comment that I might wish to downvote.
It's a neat project. It's a shame people will pile-on over your unfortunate choice of title here. Your "Important note" on github is pretty clear, I think.
Yeah, I suspect all of the comments will either be about why somebody thinks the algorithm is stupid or why they don't think they need a filter on HN or why they think filters in general are stupid or promote groupthink or whatever.
The author seems pretty clear that this was just a programming exercise written for fun and self-education and makes no claims about its perfection.
As a side note, I agree that "fanboy" and all of its permutations is a 99.9% reliable indication of a worthless comment (although such a filter would obviously catch this post :). This is regardless of whether the poster is referring to themselves or someone else or some hypothetical group that they imagine exists somewhere, and regardless of the subject of the fanboyism.
I hope you don't include my comment above in that 'pile-on'. I had no intention of negative criticism on the project or the author; just wanted to give a quick tl;dr of how this library works. Nothing more, nothing less.
I think I'd filter by phrases such as "orders of magnitude" if the rest of the comment doesn't talk about different kinds of memory access or any other technical subject ("The days when I get up before 6 AM I am orders of magnitude more productive"). It would be more of a cliché filter than inaneness-filter, though.
Unsurprisingly, such a technique seems likely to be biased against non-native speakers. But it's entirely possible to make insightful points without sophisticated language.
Alas, it is all too feasible to compose inane comments using sesquipedalian vocabulary and an overly-reticulated sentence structure in which dependent clauses which add nothing to the comment save for syntactic complexity, and sometimes only a small measure of that, as when one just links relative clause to relative clause, are stacked, to no one's benefit, one on top of the other.
If I facepalm and close the comment tab, it is usually not because if a single posting, but because of a complete subthread that has become a trope. E.g, every time a post contains the phrase "steal software", there will be someone saying "you can only steal cars, not software" and it will go on forever; same for eating meat/going veg, is Google evil or not, skeuomorphism vs flat design. I haven't ever learned anything from these discussions.
Too bad I have already started another weekend project :)
What you have created is clearly A Very Bad Idea. A readable sentence isn't necessarily inane. And this gem really makes me laugh "Conversely, some of the most inane comments of all can escape this filter by not using any punctuation (the script will read it as one long sentence)."
But I think that is perhaps beside the point: congrats on writing your first Chrome extension!
Were I to have more patience and a digital copy of a Hemingway novel, I would be very curious to see the results. It's an interesting idea, but were you to enable it in practice, you'd lose a lot of quality content and have an HN even more dominated by native English speakers.
I sort of like it, but then I'm verbose to begin with. With tools like this, I think it's better to fold in the supposedly-offending comment as opposed to just making it disappear. Without the threading cues, some pages may become hard to read.
The whole reason why HN is my preferred place to read news, is that I do not need a filter like this. Comments are generally useful and if someone posts something stupid, its greyed out in the bottom. That being said, cool idea.
Posts like this one (https://news.ycombinator.com/item?id=5543140) are removed. That is a valuable post. It directly solves a problem, while highlighting a new feature that people not not have been aware of.
I only point this out as in the article linked to by the README says: "Just because a comment is easy to read does not mean it’s inherently worthless, and in some cases these are the most important ones."
Good first shot though. =) I wonder if you could reverse the plugin to help improve my own comments.
That said, the code looks good and the idea is fun - might be nice to try and implement a javascript NLP library to make it more intelligent? [1]
[1] http://www.chrisumbel.com/article/node_js_natural_language_n...