Advertisement
Guest User

Untitled

a guest
Jun 17th, 2019
96
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.17 KB | None | 0 0
  1. # test-conda-env-hpc
  2.  
  3. ## Question
  4.  
  5. Does the conda environment that you have activated on the login node also get used in the compute nodes when submitting a job?
  6.  
  7. ## Methods
  8.  
  9. * Used conda version 4.6.14
  10. 1. Activate a conda env with snakemake installed.
  11. 1. Submit the job & note the job id.
  12. 1. The log file should have snakemake output.
  13. 1. Activate a conda env without snakemake.
  14. 1. Submit the job & note the job id.
  15. 1. The log file should have an error message that the snakemake command was not found.
  16.  
  17. ### SLURM
  18. * HPC: `beta-login.stage.arc-ts.umich.edu`
  19. * Script: `submit-slurm.sh`
  20. * Conda environments:
  21. * With snakemake: `slurm-with-smk.yaml`
  22. * Without snakemake: `slurm-no-smk.yaml`
  23.  
  24. ### PBS-Torque
  25. * HPC: `flux-login.arc-ts.umich.edu`
  26. * Script: `submit-pbs.sh`
  27. * Conda environments:
  28. * With snakemake: `pbs-with-smk.yaml`
  29. * Without snakemake: `pbs-no-smk.yaml`
  30.  
  31. ## Results
  32.  
  33. ### SLURM job with snakemake: 33069
  34.  
  35. ```
  36. Building DAG of jobs...
  37. Using shell: /usr/bin/bash
  38. Provided cores: 1
  39. Rules claiming more threads will be scaled down.
  40. Job counts:
  41. count jobs
  42. 1 touch
  43. 1
  44.  
  45. [Mon Jun 3 14:06:20 2019]
  46. rule touch:
  47. output: file.txt
  48. jobid: 0
  49.  
  50. [Mon Jun 3 14:06:20 2019]
  51. Finished job 0.
  52. 1 of 1 steps (100%) done
  53. Complete log: /home/sovacool/test-conda-env/.snakemake/log/2019-06-03T140620.029171.snakemake.log
  54. ```
  55.  
  56. ### SLURM job without snakemake: 33070
  57.  
  58. Conda environment: `slurm-no-smk.yaml`
  59.  
  60. ```
  61. /var/spool/slurmd.spool/job33070/slurm_script: line 49: snakemake: command not found
  62. ```
  63.  
  64. ### PBS job with snakemake: 33732423
  65.  
  66. ```
  67. nyx5568
  68. /home/sovacool/test-conda-env-hpc
  69. Building DAG of jobs...
  70. Using shell: /usr/bin/bash
  71. Provided cores: 1
  72. Rules claiming more threads will be scaled down.
  73. Job counts:
  74. count jobs
  75. 1 touch
  76. 1
  77.  
  78. [Wed Jun 5 11:01:27 2019]
  79. rule touch:
  80. output: file.txt
  81. jobid: 0
  82.  
  83. [Wed Jun 5 11:01:27 2019]
  84. Finished job 0.
  85. 1 of 1 steps (100%) done
  86. Complete log: /home/sovacool/test-conda-env-hpc/.snakemake/log/2019-06-05T110127.477703.snakemake.log
  87. ```
  88.  
  89. ### PBS job without snakemake: 33732416
  90.  
  91. ```
  92. nyx5089
  93. /home/sovacool/test-conda-env-hpc
  94. /var/spool/torque/mom_priv/jobs/33732416.nyx.arc-ts.umich.edu.SC: line 21: snakemake: command not found
  95. ```
  96.  
  97. ## Conclusion
  98.  
  99. Yes!
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement