Hacker News new | ask | show | jobs
by petcat 105 days ago
> An AI pipeline breaks each one down into plain-language summaries and shows who it impacts by demographic group.

Wont this process be inherently biased by itself? Usually attempts (by humans or computers) to "summarize" or frame things in "plain language" will apply a bias since it intentionally omits all the myriad context and legal/societal "gray areas" that will inform one perspective or another.
2 comments
schreiaj 105 days ago
As someone who has been working on this space for a while (not affiliated with govbase) this is really hard. Between eliminating the sycophancy that seems baked into LLMs and dealing with generalized hallucinations - it's freaking hard. I spent this weekend trying to figure out how to get my system to stop telling me the SAVE Act would be fine because it doesn't say what the process for if birth certificate doesn't match current id.

No, I haven't found a good solution yet - I'm going down a rabbit hole of basically crawling the entire federal register for referenced legislation and then adding in an adversarial agent to see if that can spot gaps.

link
foxfoxx 105 days ago
Very true. We're constantly trying to refine this and eventually plan on hiring policy researchers for a human in the loop but we just don't have the funding for that currently. We are trying to be transparent for how our scoring does work which you can read more about here: https://govbase.com/methodology

The biggest issue we have found, as you have mentioned, is just the larger context. For example (I don't think this is a real example and would need to check), the TikTok purchase deal could be ranked as an overall benefit for gig workers making content since the outlined alternative was a flat out ban hurting their income. So a deal going through, alleviating that alternative of a ban, in a vacuum is good. However, that ignores the larger context of where that option even came from and the surrounding political context around that deal. So we know the system isn't perfect right now and we're constantly trying to optimize to get the larger picture.

link