Advertisement
Guest User

Untitled

a guest
Oct 23rd, 2017
73
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 4.22 KB | None | 0 0
  1. from __future__ import print_function
  2. import re
  3.  
  4. import nltk
  5. from nltk import word_tokenize
  6. from nltk.util import ngrams
  7. from collections import Counter
  8. from nltk.tokenize import RegexpTokenizer
  9.  
  10. sample_text = """
  11. 1609
  12.  
  13. THE SONNETS
  14.  
  15. by William Shakespeare
  16.  
  17.  
  18.  
  19.                     1
  20.  From fairest creatures we desire increase,
  21.  That thereby beauty's rose might never die,
  22.  But as the riper should by time decease,
  23.  His tender heir might bear his memory:
  24.  But thou contracted to thine own bright eyes,
  25.  Feed'st thy light's flame with self-substantial fuel,
  26.  Making a famine where abundance lies,
  27.  Thy self thy foe, to thy sweet self too cruel:
  28.  Thou that art now the world's fresh ornament,
  29.  And only herald to the gaudy spring,
  30.  Within thine own bud buriest thy content,
  31.  And tender churl mak'st waste in niggarding:
  32.    Pity the world, or else this glutton be,
  33.    To eat the world's due, by the grave and thee.
  34.  
  35.  
  36.                     2
  37.  When forty winters shall besiege thy brow,
  38.  And dig deep trenches in thy beauty's field,
  39.  Thy youth's proud livery so gazed on now,
  40.  Will be a tattered weed of small worth held:
  41.  Then being asked, where all thy beauty lies,
  42.  Where all the treasure of thy lusty days;
  43.  To say within thine own deep sunken eyes,
  44.  Were an all-eating shame, and thriftless praise.
  45.  How much more praise deserved thy beauty's use,
  46.  If thou couldst answer 'This fair child of mine
  47.  Shall sum my count, and make my old excuse'
  48.  Proving his beauty by succession thine.
  49.    This were to be new made when thou art old,
  50.    And see thy blood warm when thou feel'st it cold.
  51.  
  52.  
  53.                     3
  54.  Look in thy glass and tell the face thou viewest,
  55.  Now is the time that face should form another,
  56.  Whose fresh repair if now thou not renewest,
  57.  Thou dost beguile the world, unbless some mother.
  58.  For where is she so fair whose uneared womb
  59.  Disdains the tillage of thy husbandry?
  60.  Or who is he so fond will be the tomb,
  61.  Of his self-love to stop posterity?
  62.  Thou art thy mother's glass and she in thee
  63.  Calls back the lovely April of her prime,
  64.  So thou through windows of thine age shalt see,
  65.  Despite of wrinkles this thy golden time.
  66.    But if thou live remembered not to be,
  67.    Die single and thine image dies with thee.
  68.  
  69.  
  70.                     4
  71.  Unthrifty loveliness why dost thou spend,
  72.  Upon thy self thy beauty's legacy?
  73.  Nature's bequest gives nothing but doth lend,
  74.  And being frank she lends to those are free:
  75.  Then beauteous niggard why dost thou abuse,
  76.  The bounteous largess given thee to give?
  77.  Profitless usurer why dost thou use
  78.  So great a sum of sums yet canst not live?
  79.  For having traffic with thy self alone,
  80.  Thou of thy self thy sweet self dost deceive,
  81.  Then how when nature calls thee to be gone,
  82.  What acceptable audit canst thou leave?
  83.    Thy unused beauty must be tombed with thee,
  84.    Which used lives th' executor to be.
  85.  
  86.  
  87.                     5
  88.  Those hours that with gentle work did frame
  89.  The lovely gaze where every eye doth dwell
  90.  Will play the tyrants to the very same,
  91.  And that unfair which fairly doth excel:
  92.  For never-resting time leads summer on
  93.  To hideous winter and confounds him there,
  94.  Sap checked with frost and lusty leaves quite gone,
  95.  Beauty o'er-snowed and bareness every where:
  96.  Then were not summer's distillation left
  97.  A liquid prisoner pent in walls of glass,
  98.  Beauty's effect with beauty were bereft,
  99.  Nor it nor no remembrance what it was.
  100.    But flowers distilled though they with winter meet,
  101.    Leese but their show, their substance still lives sweet.
  102. """
  103.  
  104. if __name__ == '__main__':
  105.     line = raw_input()
  106.     word = nltk.word_tokenize(line)
  107.     length = len(nltk.word_tokenize(line))
  108.     print(word)
  109.     if length == 2:
  110.         #unigrams
  111.         tokenizer = RegexpTokenizer(r'\w{3,}')
  112.         unigrams = tokenizer.tokenize(sample_text.lower())
  113.         fdistU = nltk.FreqDist(unigrams)
  114.         for k,v in fdistU.items():
  115.             print(k, v)
  116.     elif length == 3:
  117.         #bigrams
  118.         tokenizer = RegexpTokenizer(r'\w+')
  119.         unigrams = tokenizer.tokenize(sample_text.lower())
  120.         bigrams = nltk.bigrams(unigrams)    
  121.         fdistB = nltk.FreqDist(bigrams)
  122.         for k,v in fdistB.items():
  123.             print(k, v)
  124.     else:
  125.         print("invalid input")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement