[imhoguy]

This already comes from regulators, e.g. "right to explanation" laws which require any automated legaly binding decisions to be explained.

https://en.m.wikipedia.org/wiki/Right_to_explanation