sxiii

Automatic downloading files from list with existence check

Sep 23rd, 2012
172
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 0.65 KB | None | 0 0
  1. #!/bin/bash
  2. # This script downloads files from the list with name file*.list
  3. # Each row in any of the file*.list must be full URL like
  4. # http://example.com/example.file
  5. # Also if not all of files were downloaded it can re-check and download
  6. # the files that were missed ("mirroring" function).
  7. # To make this script work dont forget to create list of URLs with name file.list
  8. # Or you may change the "listname" variable to suit your needs.
  9.  
  10. listname=file*.list
  11.  
  12. cat $listname | while read line; do
  13. let rownum=$(grep -o "/" <<<"$line" | wc -l)+1
  14. namev=$(echo $line | awk -F"/" '{ print '"$"$rownum' }')
  15. if [ ! -f $namev ]; then wget $line; fi
  16. done
Add Comment
Please, Sign In to add comment