Then what? They grew a conscience when Bush Sr became president? What was their "come to Jesus" moment that led to reform in how they conduct themselves?
Improved record keeping and legal discoverability is my guess. So shady things moved to places where they could control it better. Like foreigners detained without due process.
They wouldn't want to be avoiding only prosecution, but also whatever embarrassing details might come out. Iran-Contra, for example, was personally embarrassing for very high-level people. Lots of documents mentioning, quoting, signed by, etc, high level folks[1]. Abu Ghraib, on the other hand, was mostly pawned off on the lowest level people, as if it weren't sanctioned by the Executive Office and the CIA. Related embarrassing documents were mostly signed by lawyers without direct quotes from high-level leaders. Partially because the top-down instructions took a form that was much less discoverable/accountable.
From what I can tell, the reactions to Iran-Contra and the end of the Cold War put an end to some of the more, umm, creative approaches within the CIA.
Improved record keeping and legal discoverability is my guess. So shady things moved to places where they could control it better. Like foreigners detained without due process.