[judge2020]

Earliest I found (in terms of books with a year next to them) is from 1689:

https://books.google.com/books?id=4zJWAAAAYAAJ&newbks=1&newb...

[richardfontana]

Specifically for "out of an abundance of caution", in Google Books there begin to be usages around 1911, based on a very quick search. As noted by someone else it's a pretty common phrase among American lawyers.