Advertisement
Guest User

stemming of words

a guest
Jun 7th, 2021
95
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 0.59 KB | None | 0 0
  1. from nltk.stem import PorterStemmer
  2. from nltk.tokenize import word_tokenize
  3. from slugify import slugify
  4.  
  5.  
  6. def get_root_words(text):
  7.     results = ''
  8.     vowels = ['a', 'e', 'i', 'o', 'u', 'y']
  9.     ps = PorterStemmer()
  10.     text = slugify(text, separator=' ')
  11.     words = word_tokenize(text)
  12.     for w in words:
  13.         stem = ps.stem(w)
  14.         if stem[-1] in vowels:
  15.             stem = stem[:-1]
  16.         results += stem + ' '
  17.     return slugify(results, separator=' ')
  18.  
  19.  
  20. print(get_root_words("Toto, I have a feeling we're not in Kansas anymore."))
  21.  
  22. # tot hav feel w r not in kans anymor
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement