Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- ${SPARK_PATH}/bin/spark-submit
- --master <K8S_MASTER>
- --deploy-mode cluster
- --name spark-py-driver
- --conf spark.executor.instances=5
- --conf spark.kubernetes.container.image=spark-py:2.4.0-rc3
- --conf spark.kubernetes.driver.pod.name=spark-py-driver
- --conf spark.executor.memory=2g
- --conf spark.driver.memory=2g
- local:///opt/spark/applications/spark_submit_test_job.py
- python: can't open file '/opt/spark/applications/spark_submit_test_job.py': [Errno 2] No such file or directory
Add Comment
Please, Sign In to add comment