API tools faq
paste
Advertisement
Guest User

OpenMPI

a guest
Mar 5th, 2014
139
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 4.82 KB | None | 0 0
raw download clone embed print report
  1. i OpenMPI, How Does Work?
  2. ii Building a distributed resource cluster
  3. iii Setting up OpenMPI
  4. iiii Running the code
  5. iiiii John the Ripper with OpenMPI
  6.  
  7. i OpenMPI, How Does Work?
  8. OpenMPI itself is an open source project and an implementation of MPI-1 and MPI-2 standards.MPI is not a language, all MPI operations are expressed as functions, subroutines, or methods, according to the appropriate language bindings which for C,C++, Fortran-77, and Fortran-95, are part of the MPI standard.There are different versions of OpenMPI for different compiler families.
  9. We can start a program from a node of cluster called master and specify how many process we want to use , we will also specify the other nodes in the cluster we want to use.The program itself will start on each node of the cluster and each process will be able to communicate with another by using the MPI library and all results will be sent to that node from which this program was called (in this case to the master)
  10.  
  11. ii Building a distributed resource cluster
  12. We need at least 2 Unix/Linux nodes using the same processor architecture, each node will need to be configured with a common user name with the same configuration path.The user must be authorized to log into each node without a password authentication (This can be done by using ssh PKI authentication.)To perform this configuration I will use an 64 bit Intel architecture.I'm using Debian as a main operating system and these nodes are a part of a cluster( Morpheus 3 node Cluster build previously echelon Master ,phoenix Slave1, nexus Slave2) However the nodes can also be used as stand alone.On each node must be created the same user, (to build this i made a user named "xmpi")To allow commands to be interactively run on multiple servers over an ssh connection can be used a tool called ClusterSSH (a.k.a cssh) https://github.com/duncs/clusterssh This tool can save your time when you need to run the same command in real-time on more that one cluster node.It should be used carefully, especially when you perform commands like "rm" or "shutdown".Add users set the passwords and generate a DSA ssh key
  13.  
  14. useradd xmpi -m -b /home
  15. passwd xmpi
  16. ssh-keygen -t dsa
  17.  
  18. Note:The user will be created by root but other commands should be performed by user itself.The keys shoud be copied across all nodes to permit xmpi user accessing without password
  19.  
  20. xmpi@echelon:~$ scp /home/xmpi/.ssh/id_dsa.pub xmpi@nexus:/home/xmpi.ssh/authorized_keys
  21. xmpi@echelon:~$ scp /home/xmpi/.ssh/id_dsa.pub xmpi@phoenix:/home/xmpi.ssh/authorized_keys
  22.  
  23. iii Setting up OpenMPI
  24.  
  25. Installing the package
  26.  
  27. apt-get install openmpi-bin openmpi-common libopenmpi-dev
  28.  
  29. Debian should install all needed dependencies , this action should be performed on each node.OpenMPI can be used with languages like C,C++, Fortran and provides some wrappers that can be used to compile code.mpicc is an OpenMPI C wrapper compiler , it can be used to compile C code.OpenMPI will require ssh to connect to each node , ensure that users have the appropriate permissions.First the program must be compiled and replicated to each node under the same path.An Example: report.c This program will stamp a "Hello" from each process from each node of the cluster
  30.  
  31. xmpi@echelon:~$ mpicc -o report report.c
  32.  
  33. To run this program across the cluster we must also create a file containing all host names on which we want to perform this action
  34.  
  35. xmpi@echelon:~$ cat machine
  36. echelon
  37. phoenix
  38. nexus
  39.  
  40. iiii Running the code
  41.  
  42. mpirun --mca btl ^openib -np 10 -machinefile machine report
  43.  
  44. "-np" Run this many copies of the program on the given nodes. This option indicates that the specified file is an executable program and not an application context."--mca btl ^openib" since the cluster don't have OpenFabrics expansion card support I'm disabling the openib option to suppress the errors.
  45.  
  46. OpenMPI Parallel and distributed processing
  47. OpenMPI Parallel and distributed processing
  48.  
  49. iiiii John the Ripper with OpenMPI?
  50.  
  51. Googling i found some some custom builds of John the Ripper http://openwall.info/wiki/john/custom-builds which supports parallel and distributed processing I downloaded the latest john-1.7.9-jumbo-7-Linux-x86-64 and made some Benchmarking
  52.  
  53. Without parallelization
  54. Benchmarking: dynamic_10: md5($s.md5($s.$p)) [128/128 SSE2 intrinsics 10x4x3]... DONE
  55. Many salts: 1543K c/s real, 1543K c/s virtual
  56. Only one salt: 1307K c/s real, 1307K c/s virtual
  57.  
  58. Enabling supports parallel and distributed processing using OpenMPI
  59. cat nodes.txt
  60. echelon slots=2
  61. phoenix slots=2
  62. mpirun -np 12 -hostfile nodes.txt ./john --test
  63.  
  64. Benchmarking: dynamic_10: md5($s.md5($s.$p)) [128/128 SSE2 intrinsics 10x4x3]... DONE
  65. Many salts: 2016K c/s real, 12732K c/s virtual
  66. Only one salt: 780720 c/s real, 4592K c/s virtual
  67.  
  68. References
  69. OpenMPI Project Homepage
  70. John the Ripper distributed processing
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!