[JoshTriplett]

That assumes you can go directly from any spot to any other spot, which you can't. Given a grid that looks like this:

    123
    456
    789

you can't go directly from 1 to 3 unless 2 has been selected. I also don't think you can immediately backtrack, as in 213; you have to go 2513, for instance.

With that in mind, I wrote a quick search:

    Adjacent only (can't go from 1 to 6 or 8)
    1 9
    2 40
    3 160
    4 496
    5 1208
    6 2240
    7 2984
    8 2384
    9 784
    total: 10305
    total with length >= 4: 10096

    Non-adjacent (knight-moves) allowed (can go from 1 to 6 or 8)
    1 9
    2 56
    3 304
    4 1400
    5 5328
    6 16032
    7 35328
    8 49536
    9 32256
    total: 140249
    total with length >= 4: 139880

    Pass through previous spots, no immediate backtracking (2513 but not 213)
    1 9
    2 56
    3 304
    4 1464
    5 6136
    6 21344
    7 57184
    8 105376
    9 100928
    total: 292801
    total with length >= 4: 292432

    Pass through previous spots, with immediate backtracking allowed (213)
    1 9
    2 56
    3 320
    4 1624
    5 7152
    6 26016
    7 72912
    8 140704
    9 140704
    total: 389497
    total with length >= 4: 389112

So, I believe the correct number is 292432. For comparison, that's less secure than a 6-digit PIN, or a 4-lowercase-letter password.

More importantly, though, if you're using an unlock pattern, you can't be using disk encryption, so anyone who has physical possession of your phone need not bother brute-forcing the unlock pattern.