liskl

parallel rsync

Sep 10th, 2014
262
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 0.66 KB | None | 0 0
  1. /bin/bash
  2.  
  3. # SETUP OPTIONS
  4. export SRCDIR="/folder/path"
  5. export DESTDIR="/folder2/path"
  6. export THREADS="8"
  7.  
  8. # RSYNC TOP LEVEL FILES AND DIRECTORY STRUCTURE
  9. rsync -lptgoDvzd $SRCDIR/ /$DESTDIR/
  10. # OVER SSH #rsync -lptgoDvzd -e 'ssh -c arcfour' $SRCDIR/ remotehost:/$DESTDIR/
  11.  
  12. # FIND ALL FILES AND PASS THEM TO MULTIPLE RSYNC PROCESSES
  13. cd $SRCDIR; find . -type f | xargs -n1 -P$THREADS -I% rsync -az % /$DESTDIR/%
  14. # OVER SSH #cd $SRCDIR; find . -type f | xargs -n1 -P$THREADS -I% rsync -az -e 'ssh -c arcfour' % remotehost:/$DESTDIR/% 
  15.  
  16. # IF YOU WANT TO LIMIT THE IO PRIORITY,
  17. # PREPEND THE FOLLOWING TO THE rsync & cd/find COMMANDS ABOVE:
  18. #   ionice -c2
Advertisement
Add Comment
Please, Sign In to add comment