[kwantam]

Maxima rocks for symbolic math. I prefer it to Mathematica, which is saying a lot. In contrast, octave always feels like "almost-Matlab" and I still prefer the latter.

Also see wxMaxima, which will (among many other things) produce LaTeX for you.

[nurbl]

If you haven't already, you might also want to check out Sage (http://www.sagemath.org/) It uses python to "glue" together other free math tools (e.g. Maxima) into a unified system with a nice interface.