Be careful of your jurisdiction. In some places, the act itself of secret recording can be a criminal offence. In that case, you will want to include the possibility of prison in your cost-benefit analysis.
In some places, secret recordings are useless as evidence (e.g., two-party consent states in the United States), while in others even illegal recordings may be admissible anyway (e.g., Sweden).
For example, in Germany, a lot of the privacy culture (and therefore, laws) came about because of how the secret police terrorised the common people, for example, using secret recordings and worse (e.g., secret evidence).
But in other places, sure, I would agree with you.
Secret recording may be a criminal offense. I am not a lawyer in any jurisdiction, but it appears to be a felony in parts of the US. Admitting to felonies in court is probably not a good legal strategy. Even if you don't get arrested I am not confident the evidence would be admissible.
EDIT: I know there are places where secret recordings are illegal but don't know the expected penalties. Recordings are probably legal in a "one-party consent" state. Be wary; certain kinds of recording may be illegal even if recording is legal in general.
In a lawsuit, you will be required to hand over all relevant records. It is illegal to destroy them or fail to hand them over. The penalties for doing so could be severe.
But even if you get away with it, hiding evidence would be wrong. Don't respond to sleaze with yet more sleaze. Maybe you'll win the fight or maybe you'll lose it, but you will always lose something of yourself.
E.g., Germany: StGB 201 ยง 201 Verletzung der Vertraulichkeit des Wortes https://www.gesetze-im-internet.de/stgb/__201.html
In some places, secret recordings are useless as evidence (e.g., two-party consent states in the United States), while in others even illegal recordings may be admissible anyway (e.g., Sweden).