spark-submit --conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=./app/myapplication/bin/python --master yarn --deploy-mode cluster --queue dev --archives /opt/myapplication.zip#app /bin/first_pipeline.py

--archives - Shipping my whole conda environment
spark.yarn.appMasterEnv.PYSPARK_PYTHON - setting python interpreter
--queue - which Yarn queue to be used
first_pipeline.py - is the file which I want to execute (this file present inside bin folder myapplication/bin/first_pipeline.py)