[msamwald]

Already working on this: https://examine.dev/

"In the examine|AI system, the base AI (e.g. ChatGPT) is continuously supervised and corrected by a supervisor AI. The supervisor can both passively monitor and evaluate the output of the base AI, or can actively query the base AI. This way, users and developers interact with the team of base and supervisor systems. Performance, robustness and truthfulness are enhaced by the automated evaluation, critique and improvement afforded by the supervisor.

Our approach is inspired by the Socratic method, which aims to identify underlying assumptions, contradictions and errors through dialog and radical questioning."