Advertisement
Guest User

Untitled

a guest
Mar 31st, 2015
230
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.76 KB | None | 0 0
  1. #!/bin/bash
  2.  
  3. echo "SPARK_WORKER_DIR=/tmp/sparktest/file/" > $SPARK_HOME/conf/spark-env.sh
  4. #echo "" > $SPARK_HOME/conf/spark-env.sh
  5.  
  6. LOCAL_HOSTNAME=`hostname`
  7.  
  8. $SPARK_HOME/sbin/start-master.sh
  9. echo "sleeping for 10 s"
  10. sleep 10
  11. # Try to remove the -d parameter and/or the leading env.var.
  12. SPARK_WORKER_DIR=/tmp/sparktest/envvar/ $SPARK_HOME/sbin/start-slave.sh 1 spark://$LOCAL_HOSTNAME:7077 -d /tmp/sparktest/cmdline/
  13.  
  14. echo "sleeping for 10 s"
  15. sleep 10
  16.  
  17. echo starting
  18. $SPARK_HOME/bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://$LOCAL_HOSTNAME:7077 $SPARK_HOME/lib/spark-examples-1.2.0-hadoop2.4.0.jar 100
  19.  
  20. echo "sleeping for 10 s"
  21. sleep 10
  22.  
  23. $SPARK_HOME/sbin/spark-daemon.sh stop org.apache.spark.deploy.worker.Worker 1
  24. $SPARK_HOME/sbin/stop-master.sh
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement