Advertisement
zeerx7

Remove duplicate urls

Nov 10th, 2020
200
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 0.33 KB | None | 0 0
  1. from urlparse import urlparse
  2. import re
  3. tmp = []
  4. print "- Coded by Zeerx7 -"
  5. x = open(raw_input('Your list: '),'r').read().split('\n')
  6. for c in x:
  7.  if re.search('http(s)?://',c):pass
  8.  else: c = 'http://'+c
  9.  f = urlparse(c).netloc
  10.  if f in tmp:pass
  11.  elif f:print urlparse(c).netloc;tmp.append(f);open("result.txt", "a").write(f+"\n")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement