Cooler_

crawler form & url test

Dec 12th, 2012
162
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Ruby 1.08 KB | None | 0 0
  1. #!/usr/bin/ruby
  2. =begin rdoc
  3. exemplo de uso da gem anemone
  4. autor: Cooler_
  5.  
  6. exemplo:
  7. Para extrair todas URLs de forma recursiva de um host
  8. ========================================================
  9. gem install anemone
  10.  
  11. =end
  12.  
  13. require 'rubygems'
  14. require 'anemone'
  15.  
  16. if(ARGV[0])
  17.  puts ARGV[0]
  18.  
  19. # soh exemplo pegando todas URLs de um host qualquer
  20.  Anemone.crawl(ARGV[0],opts ={
  21.                 :threads => 8,
  22.                 :user_agent => "Hellblazer browser , utf8 , NetBSD 4.1 ",
  23.                 :delay => 0,
  24.                 :depth_limit => false,
  25.                :accept_cookies => false,
  26.               }) do |anemone|
  27.    anemone.on_every_page do |page|
  28.      
  29.      forms= page.doc.at('form').inner_html rescue nil
  30.      puts "--------------------------------"
  31. # onde tem GET
  32.      if page.url =~ /=/
  33.        puts "GET HERE: #{page.url}"
  34.      end
  35. # onde tem formulario
  36.      if forms
  37.        puts "FORM HERE: #{page.url} "
  38.      end
  39.     end
  40.    
  41.  end
  42. else
  43. puts "recursive url extractor\nfollow example\nruby code.rb http://site.com"
  44. end
Advertisement
Add Comment
Please, Sign In to add comment