Hacker News new | ask | show | jobs
by michael_dorfman 5961 days ago
I've got mixed opinions of StackOverflow.

On the one hand, it's definitely better than ExpertsExchange, and does an adequate job of providing answers to many developer questions.

On the other hand, it's far (far!) from living up to it's potential. The basic premise of the site is that the wisdom of the crowd will separate the gold from the dross; however, the incentives lead to a plethora of mediocre answers which do not converge (Wikipedia-style) towards some platonic ideal, despite Joel's stated wishes to that effect. What's more, there's a well-documented problem ("the fastest gun in the West") whereby the first semi-correct answer will quickly rise to the top, drowning out later (but more correct, more thorough) answers.

In short: they haven't (yet) solved the basic problem of "changing the way people get answers".

What's more, gaining the critical mass of early users was leveraged largely through Joel and Jeff's celebrity. Which is not transferrable to other domains. Which means that the site can't easily scale out.

Based on the preceding, I'd say that VC money is a good way to go-- it should give them the resources to a) solve the basic problem, and b) build subscriber bases in other domains.

10 comments

It is far from perfect, yes, but it's also much better than anything that's come along before.

More and more when I search for solutions to problems I'm having I've been coming up with SO links. And when I do and the SO question is related to my issue I usually get the solution, too.

The "fastest gun in the west" bug has been fixed for awhile; answers at equivalent ratings are ordered randomly. In my experience, that's eliminated the problem 90% of the time and ameliorated it the rest.
I agree, this does help fix the "fastest gun in the west" issue, but I don't think it eliminates the problem.

A mediocre answer that was posted one minute after the question was submitted often has more votes than a well thought answer posted several hours later.

A better solution might be to randomly order answers, regardless of votes, for the first couple of hours (maybe even day). This will further help remove any ordering bias.

I also feel that authors of answers should be hidden for the first couple of hours (or day). Just because John Resig answered a question, doesn't necessarily mean it's the best response.

reddit also tried to solve this same problem with their "best" ordering: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s...
If I understand it correctly, they introduced this algorithm to solve the exact same problem. Hence, it should work beautifully for solving the "Fastest gun in the West" problem also.

StackOverflow should seriously consider using this algorithm, at least for a day since the question was asked, as one previous commenter said.

Why not hide all submitted answers until 24 hours after the question is posted? Once this interval expires, there would presumably be enough of them to be shuffled and presented without any sort of fastest-gun bias.

Optionally, question submitters could pay for "Urgent" status, where the submitted answers are posted as they come in. No ratings or votes would be accepted for the same cooling-off period, however.

I don't think you should hide the answers, but you should definitely hide the ranking (votes) for about a day (or half a day) or so. During this stage the answers would be randomly sorted. After a day you get to see the votes for each answer...
i posted a question because im stuck and i want my answer nownownow
The "fastest gun in the west" problem is really only an issue if you want an answer within one day of asking the question. Long term I haven't noticed it being a big deal. Over time, if a question is relevant, good lower ranked answers will rise to the top even if faster, lower quality answers were voted up initially.

The worst case scenario is a question which is perfectly written such that it garners attention from experienced folks who know what they're talking about for only a tiny amount of time, and ever after only garners attention (via google searches, for example) from clueless noobs who can't judge the quality of the answers (and don't have the rep. on the site to vote answers up or down anyway). However, this is a pretty contrived scenario which doesn't play out often in real life. Important questions will garner attention by people with a range of experience and qualifications, which will have the side effect of improving the quality of answers over time through future voting etc.

My problem is the mandate of the cc-wiki by attribution license, when cc licenses are not intended for code. A pessimistic reading would appear to restrict even reimplementaitons of answers.

Of course, the main problem is that individuals should be able to license their contributions as they choose.

You can relicense your own contributions as you choose, as long as it's at least as permissive as cc-by-sa or you dual license your code under cc-by-sa. Just mention your license in your user info. I license all my code samples under the WTFPLv2.

Also, it doesn't matter for most code sample on there. Most are not anything you would want to copy (too specific, or too general, to apply directly to your problem), or short enough that copying them would be considered fair use.

What's more, there's a well-documented problem ("the fastest gun in the West") whereby the first semi-correct answer will quickly rise to the top, drowning out later (but more correct, more thorough) answers.

I haven't noticed this. I know I have personally answered questions months after an answer was accepted, and have gotten plenty of points. In some instances, the "accepted answer" was even changed to mine.

Stack Overflow's problem is that 80% of the answers are from talkative people with no clue, and 100% of the questions are from people who have never heard of Google.

I think your analysis is correct, but all of of your criticisms are most likely fixable. They may need lots of tweaking, but overall the concept is solid!
I agree the problems are fixable.

The more germane question is, are the problems best fixed via an infusion of VC cash?

In this case, I think the answer is "yes".

The fastest-gun-in-the-west problem can be a feature depending on your perspective.

If I come to Stack Overflow with a question, many times a quick, off the cuff but not perfect answer will solve my problem. Sure, it might not be the best possible answer, but if it solves my problem, why would I want to look for something else? The primary person getting value out of an answer should be the person who asked the question, or else no one will bother coming to ask their questions. If you put anything in to slow down the submission of answers, you reduce the value of the site to people asking questions.

And there's an easy solution if the top rated and accepted answer is actually incorrect or misleadingly incomplete; just edit it. Sure, it takes a bit more rep to be able to edit people's answers, but it doesn't take all that long to get that rep.

Why do Wikipedia articles converge to such high-quality content but that doesn't happen with other collaboratively-edited sites like SO? Maybe it's because there's such a well-defined culture to guide high-quality editing on Wikipedia (i.e. projects, bots, etc.) that doesn't exist on other sites?
What resources are you implying will help solve the basic problem?
Yes, there is plenty of room for improvement. At least make it possible to browse the site by more than one tag and fix the search engine.
depends on your perspective. as a googler, i skim all answers and get the info i need. as an asker, i want the fastest close-enough answer possible so i can figure it out and move on.

incenting 'cleanup' (the original intent of community wiki) would be great, but i doubt it would increase their traffic/revenue very much.