[jonathf]

According to my tour guide at the Bodleian Library at Oxford University, preservation of books back in the olden days was all about reducing humidity. And the best way to achieve this before proper isolation were invented was to bring them up as high as possible. Rich people who collected books therefore always kept their books upstairs.

Also the curators of such collections are also often scholars. Old men, going up stairs to read their old books.

An hypothesis on my part of course, but to me I can se how this over generations gets morphed into "wizards in towers".