daily pastebin goal
34%
SHARE
TWEET

Untitled

a guest Feb 16th, 2019 70 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. #!/bin/bash
  2. set -x
  3.  
  4. LOCAL_DIR=/home/hadoop/yarn_logs
  5. mkdir $LOCAL_DIR
  6.  
  7. for row in $( yarn application -appStates ALL -list | awk '/application_/{print $1 "," $2}' )
  8. do
  9.     fields=($(echo $row | tr "," "\n"))
  10.     yarn logs -applicationId ${fields[0]} > $LOCAL_DIR/yarn_${fields[0]}_${fields[1]}.log
  11. done
  12. if [ -z "$(ls -A $LOCAL_DIR)" ]; then
  13.     echo "($LOCAL_DIR) is empty"
  14. else
  15.     aws s3 cp $LOCAL_DIR/*.log s3://out/
  16. fi
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top