Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Starting with a bunch of gzipped log files, like "logfile-*.gz".
- First, uncompress to stdout:
- gunzip --to-stdout logfile-*.gz
- Reduce output to just 404s (the 6th field):
- | grep " 404 "
- Extract the request path:
- | cut -d " " -f 7
- Sort lexicographically, to group identical requests:
- | sort
- Reduce duplicate lines to one line, and prefix with duplicate count:
- | uniq -c
- Sort again, numerically and in reverse order, to produce a descending list of unique 404s:
- | sort --numeric-sort --reverse
- And finally, redirect to a file:
- > unique-404s.txt
- All together then:
- gunzip --to-stdout logfile-*.gz | grep " 404 " | cut -d " " -f 7 | sort | uniq -c | sort --numeric-sort --reverse > unique-404s.txt
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement