SHARE
TWEET

Untitled

a guest Apr 25th, 2019 58 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. aws emr create-cluster --name "Test cluster ngramcount" --release-label emr-5.22.0 \
  2. --service-role EMR_DefaultRole \
  3. --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole \
  4. --log-uri s3://sy.log-uri.ngramcount --enable-debugging \
  5. --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=c4.large InstanceGroupType=CORE,InstanceCount=4,InstanceType=c4.large \
  6. --steps Type=CUSTOM_JAR,Jar=s3://sy.fcode/18645-proj3-0.1-latest.jar,Args=["-input","s3://sy.tweets10m/tweets10m.txt","-output","s3://sy.output/ngram10m","-program","ngramcount"] \
  7. --auto-terminate
  8.  
  9.  
  10. aws emr create-cluster --name "Test cluster hashtagsim" --release-label emr-5.22.0 \
  11. --service-role EMR_DefaultRole \
  12. --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole \
  13. --log-uri s3://sy.log-uri.ngramcount --enable-debugging \
  14. --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=c4.large InstanceGroupType=CORE,InstanceCount=4,InstanceType=c4.large \
  15. --steps Type=CUSTOM_JAR,Jar=s3://sy.fcode/18645-proj3-0.1-latest.jar,Args=["-input","s3://sy.tweets1m/tweets1m.txt","-output","s3://sy.output/hashtag1m","-program","hashtagsim","-tmpdir","s3://sy.tmp"] \
  16. --auto-terminate
  17.  
  18.  
  19.  
  20. TMP need to be on s3
  21.  
  22.  
  23. aws emr create-cluster --name "Test cluster hashtagsim" --release-label emr-5.22.0 \
  24. --service-role EMR_DefaultRole \
  25. --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole \
  26. --log-uri s3://sy.log-uri.hashtagsim --enable-debugging \
  27. --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=c4.large InstanceGroupType=CORE,InstanceCount=4,InstanceType=c4.large \
  28. --steps Type=CUSTOM_JAR,Jar=s3://sy.fcode/18645-proj3-0.1-latest.jar,Args=["-input","s3://sy.tweets1m/tweets1m.txt","-output","s3://sy.output/hashtag1m","-program","hashtagsim","-tmpdir","s3://sy.tmp"]
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!
 
Top