Advertisement
mahmoodn

data-analytics-steps

Feb 23rd, 2016
430
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.62 KB | None | 0 0
  1. 0-a) Assuming that the current working directory is /home/mahmood/cloudsuite_v3/
  2. 0-b) Extract data-analytics.zip
  3. unzip data-analytics.zip -d .
  4.  
  5. 1) docker images
  6. if the output is empty proceed to step3.
  7.  
  8. 2) docker rmi -f IMAGE_IDS
  9.  
  10. 3) docker load -i /home/mahmood/cloudsuite_v3/ubuntu.tar
  11.  
  12. 4) docker inspect ubuntu:14.04
  13. be sure that the CMD section is not empty (or is not NULL)
  14.  
  15. 5) vim data-analytics/dataset/Dockerfile (The first line is very important)
  16. FROM ubuntu:10.04
  17. RUN apt-get update && apt-get install wget -y
  18. # create data directory
  19. RUN mkdir /data
  20. RUN cd /data \
  21. && wget -nv http://parsa.epfl.ch/cloudsuite/software/enwiki-20100904-pages-articles1.xml.bz2
  22. VOLUME ["/data"]
  23.  
  24.  
  25. 6) docker build -t data-analytics:dataset ./data-analytics/dataset
  26. the last lines of a successful build looks like
  27. Step 5 : VOLUME /data
  28. ---> Running in c1101481f462
  29. ---> 1db2326dc82b
  30. Removing intermediate container c1101481f462
  31. Successfully built 1db2326dc82b
  32.  
  33.  
  34. 7) docker create --name data data-analytics:dataset
  35.  
  36. 8) put serf and jdk in the same directory as data-analytics/base
  37. serf_0.5.0_linux_amd64.zip
  38. jdk-7u79-b15-linux-x64.tar.gz
  39.  
  40. 9) Modify base/Dockerfile (-- means delete line and ++ means add line)
  41. -- RUN curl -Lso serf.zip https://dl.bintray.com/mitchellh/serf/0.5.0_linux_amd64.zip \
  42. -- && unzip serf.zip -d /bin \
  43. -- && rm serf.zip
  44. ++ COPY serf_0.5.0_linux_amd64.zip serf_0.5.0_linux_amd64.zip
  45.  
  46. ++ RUN unzip serf_0.5.0_linux_amd64.zip -d /bin \
  47.  
  48. ++ && rm serf_0.5.0_linux_amd64.zip
  49.  
  50. -- RUN cd $SRC_DIR && curl -LO "$JDK_URL/$JDK_VER/$JDK_VER2-linux-x64.tar.gz" -H 'Cookie: oraclelicense=accept-securebackup-cookie' \
  51. -- && tar xzf $JDK_VER2-linux-x64.tar.gz && mv jdk1* $JAVA_HOME && rm -f $JDK_VER2-linux-x64.tar.gz \
  52. ++ COPY jdk-7u79-linux-x64.tar.gz jdk-7u79-linux-x64.tar.gz
  53. ++ RUN tar xzf jdk-7u79-linux-x64.tar.gz && mv jdk1* $JAVA_HOME && rm -f jdk-7u79-linux-x64.tar.gz \
  54.  
  55.  
  56.  
  57. 10) docker build -t data-analytics:base ./data-analytics/base
  58.  
  59. 11) docker run -d -t --dns 127.0.0.1 -P --name master -h master.cloudsuite.com --volumes-from data data-analytics:master
  60.  
  61. 12) docker exec -it master bash
  62. You should now see the master's prompt like this
  63. mahmood@orca:cloudsuite_v3$ docker exec -it master bash
  64. root@master:/# ls
  65. bin boot data dev etc home lib lib64 media mnt opt proc root run r
  66. root@master:/# ls /opt
  67. new_analytic new_analytic.tar.gz
  68. root@master:/# ./run.sh
  69. Type the number of slave nodes, followed by [ENTER]:
  70. ......
  71.  
  72. 13) Enter 0 as the number of slaves (the system is in single machine mode) and then it will continue to download the huge wikipedia file
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement