Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- # test-conda-env-hpc
- ## Question
- Does the conda environment that you have activated on the login node also get used in the compute nodes when submitting a job?
- ## Methods
- * Used conda version 4.6.14
- 1. Activate a conda env with snakemake installed.
- 1. Submit the job & note the job id.
- 1. The log file should have snakemake output.
- 1. Activate a conda env without snakemake.
- 1. Submit the job & note the job id.
- 1. The log file should have an error message that the snakemake command was not found.
- ### SLURM
- * HPC: `beta-login.stage.arc-ts.umich.edu`
- * Script: `submit-slurm.sh`
- * Conda environments:
- * With snakemake: `slurm-with-smk.yaml`
- * Without snakemake: `slurm-no-smk.yaml`
- ### PBS-Torque
- * HPC: `flux-login.arc-ts.umich.edu`
- * Script: `submit-pbs.sh`
- * Conda environments:
- * With snakemake: `pbs-with-smk.yaml`
- * Without snakemake: `pbs-no-smk.yaml`
- ## Results
- ### SLURM job with snakemake: 33069
- ```
- Building DAG of jobs...
- Using shell: /usr/bin/bash
- Provided cores: 1
- Rules claiming more threads will be scaled down.
- Job counts:
- count jobs
- 1 touch
- 1
- [Mon Jun 3 14:06:20 2019]
- rule touch:
- output: file.txt
- jobid: 0
- [Mon Jun 3 14:06:20 2019]
- Finished job 0.
- 1 of 1 steps (100%) done
- Complete log: /home/sovacool/test-conda-env/.snakemake/log/2019-06-03T140620.029171.snakemake.log
- ```
- ### SLURM job without snakemake: 33070
- Conda environment: `slurm-no-smk.yaml`
- ```
- /var/spool/slurmd.spool/job33070/slurm_script: line 49: snakemake: command not found
- ```
- ### PBS job with snakemake: 33732423
- ```
- nyx5568
- /home/sovacool/test-conda-env-hpc
- Building DAG of jobs...
- Using shell: /usr/bin/bash
- Provided cores: 1
- Rules claiming more threads will be scaled down.
- Job counts:
- count jobs
- 1 touch
- 1
- [Wed Jun 5 11:01:27 2019]
- rule touch:
- output: file.txt
- jobid: 0
- [Wed Jun 5 11:01:27 2019]
- Finished job 0.
- 1 of 1 steps (100%) done
- Complete log: /home/sovacool/test-conda-env-hpc/.snakemake/log/2019-06-05T110127.477703.snakemake.log
- ```
- ### PBS job without snakemake: 33732416
- ```
- nyx5089
- /home/sovacool/test-conda-env-hpc
- /var/spool/torque/mom_priv/jobs/33732416.nyx.arc-ts.umich.edu.SC: line 21: snakemake: command not found
- ```
- ## Conclusion
- Yes!
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement