Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- #!/usr/bin/env ruby
- require 'nokogiri'
- require 'open-uri'
- require 'sequel'
- require 'sqlite3'
- require 'pry'
- require 'uri'
- require 'cgi'
- DB = Sequel.connect('sqlite:///Users/RyanOConnor/workspace/testing/clubland-2/clublandlv.sqlite')
- genre_table = DB[:genres].select_map(:genre_id)
- subgenre_table = DB[:subgenres]
- puts genre_table
- binding.pry
- =begin
- clubland = "http://www.clublandlv.com/forumdisplay.php?f=46"
- doc = Nokogiri::HTML(open(clubland))
- doc.css("#forumbits").each do |grab|
- subgenres = grab.css(".forumtitle a").map(&:text)
- subgenre_links = grab.css(".forumtitle a"){[:href]}
- subgenre_links = subgenre_links.map do |links|
- link_id = links.attr("href")
- CGI.parse(URI.parse(link_id).query)['f'].first.to_i
- end
- subgenre_table.import([:subgenre, :subgenre_id], subgenres.zip(subgenre_links))
- end
- =end
- I added a Zippyshare link that includes this script as well as the database itself so you can local test.(Just change the path to the DB in the .rb)
- http://www13.zippyshare.com/v/71604817/file.html
- Basically there is an unique ID per genre, you can see this listed in the genre_id column in the genres table.
- I want to run the commented script for each genre_id. The commented script grabs the subgenres for each genre.
- clubland = "http://www.clublandlv.com/forumdisplay.php?f=46" You can see the 46 is the unique ID for a specific forum.
Advertisement
Add Comment
Please, Sign In to add comment