techne/parallel.org

Parallel

• Pulling files from remote server with rsync
• Pushing files to remote server with rsync
• Running the same command on multiple remote hosts

Pulling files from remote server with rsync

To transfer just the files:

ssh user@remote -- find /path/to/parent/directory -type f | parallel -v -j16 rsync -Havessh -aAXP user@remote:{} /local/path

To transfer the entire directory:

echo "/path/to/parent/directory" | parallel -v -j16 rsync -Havessh -aAXP user@remote:{} /local/path

Pushing files to remote server with rsync

To transfer just the files:

find /path/to/local/directory -type f | parallel -v -j16 -X rsync -aAXP /path/to/local/directory/{} user@remote:/path/to/dest/dir

Running the same command on multiple remote hosts

parallel --tag --nonall -S remote0,remote1,remote2 uptime