Hacker News new | ask | show | jobs
by temporarely 746 days ago
This is a general cognitive problem in reducing a higher dimensional object to a lower dimensional representation for processing. For a complex entity like society and/or human wellbeing, it seems inevitable that the representation will mask important non-tangible/non-measurable dimensions of the object being considered.
1 comments

Some dimensional reduction is always possible, due to the Johnson-Lindenstrauss lemma. For example, for 8 billion data points, reducing to 1400 dimensions enables preserving distances within +- 50% (that can probably be tightened a bit) regardless of what the uncompressed data is.