Hacker News new | ask | show | jobs
Gödel's Incompleteness vs. Artificial General Intelligence (AGI) in Law
2 points by ikifor 942 days ago
Internally, any AGI is a sequence of numbers. This sequence can be visible, i.e., “open source,” or hidden. The former implies an anarchy, while the latter a hierarchy.

Anarchies cannot generate income. Tyrannies are hierarchies that are severely conditioned on finances. Therefore, only a hidden or “secretive” AGI can be a tyrant. The incentives for creating such an AGI are immense (~$100T???) for its “Maker.” In the context of law, lawsuits by “Takers” are the sole but significant and material threats to the Maker. Any competent AGI is also a supreme legal expert by definition. The Maker and all Takers will recursively rely on the “legal expertise” of the AGI.

This scenario repeats the Hilbert-Gödel “We must know. We will know” duel that was decided before computers were invented. Hilbert already promised a “symbolic” AGI. Gödel proved that it could not be complete (by having unprovable truths) nor provably consistent (by having contradictions). Gödel relied on self-referencing, or recursion, perhaps the most dominant concept in computers and, thus, any AGIs.

Undoubtedly, a Maker’s absolute priority will be to defend itself from lawsuits. And the AGI, by definition, will be able to adequately verify (or at least enumerate) any fact, rule, theory, etc., about the law. For a Maker, mere verification (or enumeration) suffices to train the AGI to (recursively) protect from all lawsuits.

As a (universal approximation) function generator, any AGI is a consistent formal system implementing axiomatic theories. It is also able to carry out basic arithmetic. Therefore, it is governed by Gödel’s Incompleteness Theorems.

For the Maker, this means that the AGI will enumerate such potential lawsuits that it, the AGI, will be unable to tell whether they are winnable or not for the Maker (i.e., “provable” based on the Constitution, all statutes, and all existing legal opinions/precedents). As there is no guarantee that the Maker is wiser than all potential Takers using the same AGI, the incentivized Maker’s only option is to change the law.

1 comments

Theoretical Computer Scientist here: I don't what what "AGI" is, but neither the Large Language Models (ChatGPT) nor humans are a "a consistent formal system implementing axiomatic theories". ChatGPT cannot even reliably do arithmetic.
I was having issues editing the post. I was referring to AGIs as universal approximation functions (or generators thereof).