You can check by looking at the average comment scores on http://news.ycombinator.com/leaders (the second column). They haven't changed much as far as I can tell.
They probably wouldn't change much if the commenter had a long history to be averaged in. You'd want to look at the delta of the delta (2nd order stats) to see if any noticeable effect was occurring.
Hold on, that begs the question -- how do we know who deserves? that is the problem.
The whole thing has a circular dependency: if ranking acts as a filter, then higher rank means more readers, and that in turn means more upvotes and so a higher ranking. It is probably not so much the top being undeserving, but that deserving stuff gets missed.
This seems to be a fundamental weakness in all similar 'ranking' systems, but I am not sure of the full character and implications...
(There is certainly a substantial component of being an automated system of 'social-proof'. And the filtering can never be entirely effective: if you show everyone only the good stuff, the filtering would not get done at all.)
The advantage here is that pg is giving an advantage to high averages, not high point totals. So there's less incentive to be a reddit/digg style poweruser looking for one upvote on thousands of comments, and more incentive to say something truly insightful from time to time.
boobie testucles boobie testucles boobie testucles
One thing I like about hacker news is that some complete moron, like me, can write 'boobie testucles' in reply to the topmost comment, thus keeping the reply visible to all on top even if its voted down by the retarded masses who don't know what a good reply really is. Man, lisp sucks. Please vote my reply up, thanks!
You make a reasonable point in an, uh, illustrative way (they say "show, don't tell", right?). The vertical arrangement of comments by preorder traversal of comment trees gives some low quality comments "unfair" visibility. I don't see any solution (that maintains readability) apart from moving into a second or third dimension. The question is, is it a big enough problem to merit such a drastic response? Probably not.
Does this create a rich-get-richer type scenario, where users with high averages get more exposure, thus leading to higher averages?