Advertisement
Guest User

Untitled

a guest
Aug 8th, 2015
328
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.95 KB | None | 0 0
  1. # CKAN 6 - Virtualbox - VISTA
  2. These instructions are a CentOS 6 adapted version of CKAN’s Ubuntu install.
  3. http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html
  4.  
  5. These instructions are targeted for a Virtualbox install on StatCan’s VISTA SDCs and the version of the software packages installed.
  6.  
  7. ## Setup the environment 
  8. ### Downloading the OS
  9. Download the first DVD of latest 32 bit (i386) CentOS 6 ISO.
  10.  
  11. At the time of writing this the file required was [CentOS-6.6-i386-bin-DVD1.iso](http://isoredirect.centos.org/centos/6/isos/i386/)
  12.  
  13. ### Installing in virtuabox
  14. Create a new server and accept all defaults with the exception of memory ( use 2 GB ) and storage ( at least 10 GB ). Follow through with the install without network connectivity.
  15.  
  16. 1. Start the blank box
  17. 2. Select the image you downloaded
  18. 3. Accept defaults except:
  19. * Storage Device Warning : **[Yes, discard any data]**
  20. * Which type of installation would you like? **Use all the space**
  21. 4. Once CentOS has formatted the disk it runs to the end, ~45
  22.  
  23. ### Setup the proxy (if required)
  24. #### /etc/environment
  25. As root add the following to the bottom of the file replacing **\<user>** and **\<pass>** with the proxy credentials.
  26.  
  27. ```
  28. http_proxy="http://<user>:<pass>@stcweb.statcan.ca:80/"
  29. HTTP_PROXY="http://<user>:<pass>@stcweb.statcan.ca:80/"
  30. https_proxy="https://<user>:<pass>@stcweb.statcan.ca:80/"
  31. HTTPS_PROXY="https://<user>:<pass>@stcweb.statcan.ca:80/"
  32. no_proxy="localhost,127.0.0.1,127.0.0.1/8,/var/run/docker.sock"
  33. ```
  34.  
  35. #### /etc/apt/apt.conf.d/77proxy
  36. As root add the following to the bottom of the file replacing **\<user>** and **\<pass>** with the proxy credentials.
  37.  
  38. ```
  39. Acquire::http::proxy "http://<user>:<pass>@stcweb.statcan.ca:80/";
  40. Acquire::https::proxy "https://<user>:<pass>@stcweb.statcan.ca:80/";
  41. ```
  42.  
  43. #### Native OS proxy config [GUI]
  44. ```
  45. SYSTEM_TOOLBAR > System > Preferences > Network Proxy
  46. Proxy Configuration > Manual proxy configuration
  47.  
  48. HTTP Proxy: stcweb.statcan.ca
  49. Port: 80
  50.  
  51. [Details] > Use Authentication
  52.  
  53. Username: <user>
  54. Password: <pass>
  55. [Close]
  56.  
  57. Secure HTTP proxy: stcweb.statcan.ca
  58. Port: 80
  59. [Close]
  60. ```
  61.  
  62. #### Apply proxy settings to core applications
  63. ##### /etc/yum.conf
  64. As root add the following directly after [main] replacing **\<user>** and **\<pass>** with the proxy credentials.
  65. ```
  66. proxy=http://stcweb.statcan.ca:80/
  67. proxy_username=<user>
  68. proxy_password=<pass>
  69. ```
  70.  
  71. ##### Firefox [GUI]
  72. ```
  73. APPLICATION_TOOLBAR > Edit > Preferences
  74.  
  75. Advanced > Network > [Settings...]
  76. Use system proxy settings
  77. [Ok]
  78.  
  79. [Close]
  80. ```
  81.  
  82. #### Restart the machine
  83. To account for some odd behaviour:
  84.  
  85. 1. Close all applications ( terminals and firefox )
  86. 2. Restart the machine
  87. 3. Open firefox first, connect to an external site (inputting user:pass credentials)
  88. 4. Open terminal and continue
  89.  
  90. ### Update the current system software
  91. ```
  92. sudo yum update -y
  93. ```
  94.  
  95. ## Install CKAN
  96. Following:
  97. http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html
  98.  
  99. ### 1. Install the required packages
  100. Replace the provided by with the following:
  101.  
  102. ```
  103. rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
  104. yum install -y python-devel python-pip python-virtualenv
  105. yum install -y postgresql postgresql-devel postgresql-server
  106. yum install -y git-core
  107. yum install -y java-1.6.0-openjdk
  108. yum install -y openssl-devel
  109. yum install -y gcc
  110.  
  111. chkconfig postgresql on
  112. service postgresql initdb
  113. service postgresql start
  114. ```
  115.  
  116. #### Notes
  117. * if there are pg_config issues you may need to:
  118. ```
  119. export PATH=$PATH:/usr/pgsql-x.x/bin
  120. ```
  121.  
  122. ### 2. Install CKAN into a Python Virtual Environment (VE)
  123. As a local user, not root, execute:
  124.  
  125. #### Section a — Create a Python VE
  126. Run as is without the developer’s symlinks
  127. ```
  128. sudo mkdir -p /usr/lib/ckan/default
  129. sudo chown `whoami` /usr/lib/ckan/default
  130. virtualenv --no-site-packages /usr/lib/ckan/default
  131. . /usr/lib/ckan/default/bin/activate
  132. ```
  133.  
  134. If using a proxy and prior to installing into the python virtualenv it starts WITHOUT the proxy environment you previously setup. pip respects the lowercase variables, as such export the following:
  135. ```
  136. export http_proxy="http://<user>:<pass>@stcweb.statcan.ca:80/"
  137. export https_proxy="https://<user>:<pass>@stcweb.statcan.ca:80/"
  138. ```
  139.  
  140. #### Section b — Install CKAN in your VE
  141. Install stable CKAN 2.3
  142.  
  143. ```
  144. pip install -e 'git+https://github.com/open-data/ckan.git@canada-v2.3#egg=ckan'
  145. ```
  146.  
  147. #### Section c — Install required Python modules into the VE
  148. If behind a proxy there are SSL issues with this concoction of OS/Python versions behind the StatCan ( general? ) proxy. Ian located libraries to bridge that gap:
  149.  
  150. ```
  151. pip install pyasn1
  152. pip install ndg-httpsclient
  153. pip install pyOpenSSL
  154. ```
  155.  
  156. Continue with the normal instructions:
  157.  
  158. ```
  159. pip install -r /usr/lib/ckan/default/src/ckan/requirements.txt
  160. ```
  161.  
  162. ##### Notes
  163. * Run behind a proxy creates a series of issues
  164. * Run without a proxy it failed on Pygments==1.6, re-runing the command cleared the Pygments issue
  165. * Run with our without proxy
  166.  
  167. #### Section d — Deactivate and reactivate your VE
  168. ```
  169. deactivate
  170. . /usr/lib/ckan/default/bin/activate
  171. ```
  172.  
  173. ### 3. Setup a PostgreSQL database
  174. Same as source.
  175. #### Check for utf-8
  176. ```
  177. sudo -u postgres psql -l
  178. ```
  179. #### Create PG user
  180. ```
  181. sudo -u postgres createuser -S -D -R -P ckan_default
  182. ```
  183. #### Create the DB
  184. Be sure to save the password you use for the following command.
  185.  
  186. ```
  187. sudo -u postgres createdb -O ckan_default ckan_default -E utf-8
  188. ```
  189. #### /var/lib/pgsql/data/pg_hba.conf
  190. ##### If a local dB
  191. sqlalchemy wants IPv6, we should want to be restrictive on permissions.
  192.  
  193. 1. Disable all forms of access to the dB
  194. 2. Add ckan_default IPv6
  195. 3. Add ckan_default IPv4 (if required for alternate scripting)
  196. 4. Restart PostgreSQL
  197.  
  198. ```
  199. # TYPE DATABASE USER CIDR-ADDRESS METHOD
  200. # "local" is for Unix domain socket connections only
  201. #local all all ident
  202. # IPv4 local connections:
  203. #host all all 127.0.0.1/32 ident
  204. # IPv6 local connections:
  205. #host all all ::1/128 ident
  206. # CKAN Specific Permissions
  207. ##### required for sqlalchemy
  208. host ckan_default all ::1/128 md5
  209. ##### required for script XYZ.py
  210. #host ckan_default all 127.0.0.1/32 md5
  211. ```
  212.  
  213. ```
  214. sudo service postgresql restart
  215. ```
  216.  
  217. #### If a remote dB
  218. THIS MAY BE FALSE, SQLALCHEMY SEEMS TO DEMAND IPV6
  219. ##### postgresql.conf
  220. ```
  221. listen_addresses = 'localhost,192.168.#.#’
  222. ```
  223. ##### pg_hba.conf
  224. Consider that the following config is invalid, you may need the IPv6 equivalent.
  225. ```
  226. host    all             all             192.168.#.#/32                 md5
  227. ```
  228.  
  229. ### 4. Create a CKAN config file
  230. Same as source.
  231.  
  232. #### Create a directory to contain the site’s config files:
  233. ```
  234. sudo mkdir -p /etc/ckan/default
  235. sudo chown -R `whoami` /etc/ckan/
  236. ```
  237.  
  238. #### Change to the ckan directory and create a CKAN config file:
  239. ```
  240. cd /usr/lib/ckan/default/src/ckan
  241. paster make-config ckan /etc/ckan/default/development.ini
  242. ```
  243.  
  244. #### /etc/ckan/default/development.ini file
  245. ##### sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default
  246. Replace **pass** with the password the database used in **step 3 - Create the DB**
  247. If a remote host replace **localhost** with the remote and append **?sslmode=disable**
  248.  
  249. ###### ckan.site_id = default
  250. Replace **default** with a value unique to this install
  251.  
  252. ### 5. Setup Solr
  253. We are using an archived version of solr, 3.5, and as such we have to get it from the archives.
  254.  
  255. #### Fetch
  256. ```
  257. mkdir ~/src;cd ~/src
  258. wget https://archive.apache.org/dist/lucene/solr/3.5.0/apache-solr-3.5.0.tgz
  259. ```
  260. #### Open and move
  261. ```
  262. tar -xvzf apache-solr-3.5.0.tgz
  263. sudo mkdir /etc/apache-solr/
  264. sudo cp -R apache-solr-3.5.0/* /etc/apache-solr/
  265. ```
  266. #### Create and set permissions on the data directory
  267. ```
  268. sudo mkdir -p /etc/apache-solr/example/solr/data/
  269. sudo chown -R `whoami` /etc/apache-solr/example/solr/data/
  270. ```
  271. #### Cleanup
  272. ```
  273. rm -rf apache-solr-3.5.0*
  274. ```
  275. #### Test
  276. ```
  277. cd /etc/apache-solr/example
  278. java -jar start.jar
  279. ```
  280.  
  281. In a Browser try:
  282. [http://127.0.0.1:8983/solr/](http://127.0.0.1:8983/solr/)
  283.  
  284. #### Set startup Script
  285.  
  286. ```
  287. sudo cat > /etc/init.d/solr
  288. #!/bin/sh
  289.  
  290. # Starts, stops, and restarts Apache Solr.
  291. #
  292. # chkconfig: 35 92 08
  293. # description: Starts and stops Apache Solr
  294.  
  295. SOLR_DIR="/etc/apache-solr/example"
  296. JAVA_OPTIONS="-Xmx1048m -DSTOP.PORT=8984 -DSTOP.KEY=SHAZBOT -jar start.jar"
  297. LOG_FILE="/var/log/solr.log"
  298. JAVA="/usr/bin/java"
  299.  
  300. case $1 in
  301. start)
  302. echo "Starting Solr"
  303. cd $SOLR_DIR
  304. $JAVA $JAVA_OPTIONS 2> $LOG_FILE &
  305. ;;
  306. stop)
  307. echo "Stopping Solr"
  308. cd $SOLR_DIR
  309. $JAVA $JAVA_OPTIONS --stop
  310. ;;
  311. restart)
  312. echo “Restarting in 5s“
  313. $0 stop
  314. sleep 5
  315. $0 start
  316. ;;
  317. *)
  318. echo "Usage: $0 {start|stop|restart}" >&2
  319. exit 1
  320. ;;
  321. esac
  322. ^d
  323.  
  324. sudo chmod 755 /etc/init.d/solr
  325. sudo chkconfig --add solr
  326. sudo service solr stop
  327. sudo service solr start
  328. ```
  329.  
  330. #### Apply Schema
  331. Consider which schema you need to use, assuming it’s in the Github account
  332.  
  333. ```
  334. sudo service solr stop
  335. cd /etc/apache-solr/example/solr/conf
  336. sudo mv schema.xml schema.xml.yyyymmdd
  337. sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml schema.xml
  338. sudo service solr start
  339. ```
  340.  
  341. ### 6. Create database tables
  342.  
  343. ```
  344. cd /usr/lib/ckan/default/src/ckan
  345. paster db init -c /etc/ckan/default/development.ini
  346. ```
  347.  
  348. ### 7. Set up the DataStore
  349. skipped
  350.  
  351. ### 8. Link to who.ini
  352. ```
  353. ln -s /usr/lib/ckan/default/src/ckan/who.ini /etc/ckan/default/who.ini
  354. ```
  355.  
  356. ### 9. You’re done!
  357. ```
  358. cd /usr/lib/ckan/default/src/ckan
  359. paster serve /etc/ckan/default/development.ini
  360. ```
  361.  
  362. In a Browser try:
  363. [http://127.0.0.1:5000/](http://127.0.0.1:5000/)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement