Guest User

Untitled

a guest
Jun 25th, 2018
85
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.55 KB | None | 0 0
  1. conf = SparkConf().setAppName("product_recommendation-server")
  2. .set('spark.jars', 'path/to/elasticsearch-hadoop-2.1.0.Beta2.jar')
  3. .set('spark.driver.memory', '2250m')
  4. .set('spark.sql.shuffle.partitions', '2000')
  5.  
  6.  
  7. sc = SparkContext(conf=conf)
  8. spark = SparkSession(sc)
  9. sql_sc = sql_sc = SQLContext(sc)
  10.  
  11. path ='path/to/data.csv'
  12.  
  13. sql_sc.read.format("org.elasticsearch.spark.sql").format("org.elasticsearch.spark.sql")
  14. .load(path)
Add Comment
Please, Sign In to add comment