
Untitled
By: a guest on
Apr 30th, 2012 | syntax:
None | size: 1.50 KB | hits: 27 | expires: Never
RSS feed scraping with Python
from urllib import urlopen
from BeautifulSoup import BeautifulSoup
import re
source = urlopen('http://feeds.huffingtonpost.com/huffingtonpost/raw_feed').read()
title = re.compile('<title>(.*)</title>')
link = re.compile('<link>(.*)</link>')
find_title = re.findall(title, source)
find_link = re.findall(link, source)
literate = []
literate[:] = range(1, 16)
for i in literate:
print find_title[i]
print find_link[i]
>>> link = re.compile('<link rel="alternate" type="text/html" href=(.*)')
>>> find_link = re.findall(link, source)
>>> find_link[1].strip()
'"http://www.huffingtonpost.com/andrew-brandt/the-peyton-predicament-pa_b_1271834.html" />'
>>> len(find_link)
15
>>>
#!/usr/bin/env python
import feedparser # pip install feedparser
d = feedparser.parse('http://feeds.huffingtonpost.com/huffingtonpost/latestnews')
# .. skipped handling http errors, cacheing ..
for e in d.entries:
print(e.title)
print(e.link)
print(e.description)
print("n") # 2 newlines
Even Critics Of Safety Net Increasingly Depend On It
http://www.huffingtonpost.com/2012/02/12/safety-net-benefits_n_1271867.html
<p>Ki Gulbranson owns a logo apparel shop, deals in
<!-- ... snip ... -->
Christopher Cain, Atlanta Anti-Gay Attack Suspect, Arrested And
Charged With Aggravated Assault And Robbery
http://www.huffingtonpost.com/2012/02/12/atlanta-anti-gay-suspect-christopher-cain-arrested_n_1271811.html
<p>ATLANTA -- Atlanta police have arrested a suspect
<!-- ... snip ... -->