Advertisement
Guest User

Untitled

a guest
Oct 14th, 2019
79
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.32 KB | None | 0 0
  1. #1
  2. pyspark --jars file1.jar,file2.jar
  3.  
  4. #2
  5. ./bin/spark-submit --jars xxx.jar your_spark_script.py
  6.  
  7. #3
  8. SPARK_CLASSPATH='/path/xxx.jar:/path/xx2.jar' your_spark_script.py
  9.  
  10. #4 using spark context
  11.  
  12. conf = SparkConf().set("spark.jars", "/path-to-jar/spark-streaming-kafka-0-8-assembly_2.11-2.2.1.jar")
  13.  
  14. sc = SparkContext( conf=conf)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement