[impendia]

Neither calculus nor Epp's book require any prereqs beyond high school algebra. Studying algorithms would definitely be easier if you'd gotten through Epp first.

For linear algebra, it depends heavily on your choice of book. To read Hefferon or (even more so) Axler, you'll want to have seen a fair amount of mathematical formalism, and read through some proofs. Knop is probably a good place to learn mathematical formalism; if you approach it with limited background it will be slow going but you'll learn a lot.