Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- .union(transitive)
- .union(family)
- .groupByKey(_.key)
- .reduceGroups((left, right) =>
- "Step5_Spark_Command": "command-runner.jar,spark-submit,--class,com.ms.eng.link.modules.linkmod.Links,--name,\"Links\",--master,yarn,--deploy-mode,client,--executor-memory,32G,--executor-cores,4,--conf,spark.sql.shuffle.partitions=2020,/home/hadoop/linking.jar,jobId=#{myJobId},environment=prod",
Add Comment
Please, Sign In to add comment