[kraemate]

Does rsync auto-resume after failed connections?

[CarolineW]

I've written a script that every minute tests to see if the appropriate rsync command is running. If not, it simply runs it again. In that way it effectively restarts and gracefully resumes.

This Google search:

https://www.google.co.uk/search?q=rsync+auto-restart+failed+...

returns this link:

http://superuser.com/questions/302842/resume-rsync-over-ssh-...

which contains this script:

    #!/bin/bash

    while [ 1 ]
    do
        rsync -avz --partial source dest
        if [ "$?" = "0" ] ; then
            echo "rsync completed normally"
            exit
        else
            echo "Rsync failure. Backing off and retrying..."
            sleep 180
        fi
    done

The comment says:

    When the connection dies, rsync will quit
    with a non-zero exit code. This script simply
    keeps re-running rsync, letting it continue
    until the synchronisation completes normally.

That's pretty much what I've done.

[kraemate]

Wow, thanks for the script. Surprising in its simplicity. I would have thought this use-case was popular enough to warrant specialized tools etc. Especially in the scientific community where they transfer large files.

[CarolineW]

It's simple enough that it's the sort of thing I type out in 30 seconds and there it is. No need for specialist tools - finding the tool, remembering how to use it, working out the right parameters ...

Easier, faster, and more flexible just to write the script. It's what I do.

[atsaloli]

Yes, it it does, if you use:

   --partial               keep partially transferred files

   --append                append data onto shorter files

[Piskvorrr]

You mean, if the connection fails before completing? I don't think so (I believe that's a feature).

For automated transfers ("retry until done"), I use the lsyncd wrapper (which also watches the source files, so you don't need to poll for changes: it wakes up by inotify).

[atsaloli]

Sure, just use --partial switch:

   --partial               keep partially transferred files

[Piskvorrr]

That is incorrect. This will keep partial transfers in destination (and when invoked again, will continue tranferring from that point onwards), but will not restart an aborted transfer (e.g. for a broken connection).

In other words, this feature is a prerequisite for auto-resume, not auto-resume itself (which can be scripted in ~10 lines, as shown in a sibling thread).

[atsaloli]

Thanks! But no need to script, you can just use

            --append                append data onto shorter files

to resume interrupted transfers.