"The Policeman's Beard is Half Constructed," written in 1983 by a program named Racter. Bestseller? It's very well known, but I don't know how well it sold.
Racter was more of a William S. Burroughs cut-up program, it mostly randomizes and regurgitates it's input. Policeman's Beard required carefully selected input and is on the border of being a hoax.
This is an interesting web page that convincingly makes the case that "The Policeman's Beard is Half Constructed" was written largely by the humans involved, with the program having a minimal role, and that the Racter program they sold to the public was incapable of reproducing the novel.
> "The Library of Babel" (Spanish: La biblioteca de Babel) is a short story by Argentine author and librarian Jorge Luis Borges (1899–1986), conceiving of a universe in the form of a vast library containing all possible 410-page books of a certain format.