Having no money in your bank account can definitely impact your feeling of self worth. It can make a lot of other people angry at you, like landlords, power companies, etc...
Money is a pretty objective assessment what's a person's worth. It's an actual measurable quantity, it's meaning doesn't vary from person to person like it does other indicators of worth (i.e looks, personality, altruism etc.)
I'm not sure anything about the question at hand, a person's worth, is objective. Money is easy enough to measure and compare though, and I think that's part of it's appeal as a measure. It's precise even if it's not accurate.