SHOW:
|
|
- or go back to the newest paste.
1 | #!/usr/bin/env ruby | |
2 | require 'nokogiri' | |
3 | require 'open-uri' | |
4 | require 'sequel' | |
5 | require 'sqlite3' | |
6 | require 'pry' | |
7 | require 'uri' | |
8 | require 'cgi' | |
9 | ||
10 | ||
11 | DB = Sequel.connect('sqlite:///Users/RyanOConnor/workspace/testing/clubland-2/clublandlv.sqlite') | |
12 | genre_table = DB[:genres].select_map(:genre_id) | |
13 | subgenre_table = DB[:subgenres] | |
14 | ||
15 | puts genre_table | |
16 | binding.pry | |
17 | ||
18 | =begin | |
19 | clubland = "http://www.clublandlv.com/forumdisplay.php?f=46" | |
20 | ||
21 | doc = Nokogiri::HTML(open(clubland)) | |
22 | ||
23 | doc.css("#forumbits").each do |grab| | |
24 | subgenres = grab.css(".forumtitle a").map(&:text) | |
25 | subgenre_links = grab.css(".forumtitle a"){[:href]} | |
26 | ||
27 | subgenre_links = subgenre_links.map do |links| | |
28 | link_id = links.attr("href") | |
29 | CGI.parse(URI.parse(link_id).query)['f'].first.to_i | |
30 | end | |
31 | ||
32 | subgenre_table.import([:subgenre, :subgenre_id], subgenres.zip(subgenre_links)) | |
33 | ||
34 | end | |
35 | =end | |
36 | ||
37 | ||
38 | I added a Zippyshare link that includes this script as well as the database itself so you can local test.(Just change the path to the DB in the .rb) | |
39 | - | http://www48.zippyshare.com/v/33147561/file.html |
39 | + | http://www13.zippyshare.com/v/71604817/file.html |
40 | ||
41 | Basically there is an unique ID per genre, you can see this listed in the genre_id column in the genres table. | |
42 | I want to run the commented script for each genre_id. The commented script grabs the subgenres for each genre. | |
43 | ||
44 | clubland = "http://www.clublandlv.com/forumdisplay.php?f=46" You can see the 46 is the unique ID for a specific forum. |