[graue]

That seems to only provide the newest 20 posts.

If RSS went all the way back to the beginning, then you'd be right, it would get you 99% of the way there.

[icebraining]

You can append /rss to the pages, e.g. http://[blog].tumblr.com/page/2/rss

It'd be nice if they supported <link rel="next"> tags, but it's still extremely easy to dump the content.