[lmm]

I don't know about mathematica specificly, but the standard way is lazily, as you say. You pass around a lazy list of digits (i.e. a partial list and a function that knows how to compute the next digit and the next function); it's trivial to add or subtract two such things, multiplication and division are harder but not too bad, and once you've got the idea square root and other more complex operations are pretty straightforward.