I wonder if instead of a number, users are told what their "karmic trend" is.
Like if your last three comments got upvotes, it would say "you're trending good" or something (and vice versa for downvotes). People's monkey brains get too focused on a single number which inevitably gets tied to their identity/reputation, even on a place like HN where it's essentially invisible.
I wouldn't lose a wink if you decided to hide the karma system completely from us HN users, but I'm sure some would.