If the comment was really sparkling, it seems unlikely that the first two ignoramuses would drown out the larger cast of HN. It's not all about timing.
What if we don't assume every vote is equal. What about weighting a user's vote by some factor of their karma? And/or capping the threshold that a user can vote. i.e A user with 0 karma can vote on comments with 0-3 points. A person with 100 karma can vote on comments -3-6 points. A person with 500 karma can vote on comments -9-9 points, etc... This implies that a comment with ridiculous votes (either positive or negative) has been decided by members of the community that the community has already shown approval for.
Also, 10 people on a site that gets in the area of 200,000 uniques isn't that significant.