API tools faq
paste
lamiastella

gensim_16CPUs

lamiastella
Oct 4th, 2016
271
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 12.39 KB | None | 0 0
raw download clone embed print report
  1. jalal@klein:~/computer_vision/image_retrieval$ python gensim_test.py
  2. 2016-10-04 17:46:44,402 : INFO : collecting all words and their counts
  3. 2016-10-04 17:46:44,403 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types
  4. 2016-10-04 17:46:49,775 : INFO : collected 253854 word types from a corpus of 17005207 raw words and 1701 sentences
  5. 2016-10-04 17:46:50,150 : INFO : min_count=5 retains 71290 unique words (drops 182564)
  6. 2016-10-04 17:46:50,150 : INFO : min_count leaves 16718844 word corpus (98% of original 17005207)
  7. 2016-10-04 17:46:50,359 : INFO : deleting the raw counts dictionary of 253854 items
  8. 2016-10-04 17:46:50,390 : INFO : sample=0.001 downsamples 38 most-common words
  9. 2016-10-04 17:46:50,390 : INFO : downsampling leaves estimated 12506280 word corpus (74.8% of prior 16718844)
  10. 2016-10-04 17:46:50,390 : INFO : estimated required memory for 71290 words and 100 dimensions: 92677000 bytes
  11. 2016-10-04 17:46:50,589 : INFO : resetting layer weights
  12. 2016-10-04 17:46:51,290 : INFO : training model with 16 workers on 71290 vocabulary and 100 features, using sg=0 hs=0 sample=0.001 negative=5
  13. 2016-10-04 17:46:51,290 : INFO : expecting 1701 sentences, matching count from corpus used for vocabulary survey
  14. 2016-10-04 17:46:52,300 : INFO : PROGRESS: at 1.00% examples, 621533 words/s, in_qsize 31, out_qsize 0
  15. 2016-10-04 17:46:53,310 : INFO : PROGRESS: at 2.30% examples, 711079 words/s, in_qsize 29, out_qsize 1
  16. 2016-10-04 17:46:54,337 : INFO : PROGRESS: at 3.52% examples, 722178 words/s, in_qsize 31, out_qsize 2
  17. 2016-10-04 17:46:55,352 : INFO : PROGRESS: at 4.68% examples, 719469 words/s, in_qsize 32, out_qsize 0
  18. 2016-10-04 17:46:56,372 : INFO : PROGRESS: at 5.98% examples, 738191 words/s, in_qsize 31, out_qsize 1
  19. 2016-10-04 17:46:57,373 : INFO : PROGRESS: at 7.21% examples, 743342 words/s, in_qsize 31, out_qsize 1
  20. 2016-10-04 17:46:58,402 : INFO : PROGRESS: at 8.38% examples, 739438 words/s, in_qsize 31, out_qsize 2
  21. 2016-10-04 17:46:59,437 : INFO : PROGRESS: at 9.50% examples, 732771 words/s, in_qsize 30, out_qsize 7
  22. 2016-10-04 17:47:00,454 : INFO : PROGRESS: at 10.73% examples, 735021 words/s, in_qsize 31, out_qsize 0
  23. 2016-10-04 17:47:01,454 : INFO : PROGRESS: at 11.86% examples, 732447 words/s, in_qsize 30, out_qsize 0
  24. 2016-10-04 17:47:02,470 : INFO : PROGRESS: at 12.99% examples, 729185 words/s, in_qsize 30, out_qsize 1
  25. 2016-10-04 17:47:03,491 : INFO : PROGRESS: at 14.17% examples, 728959 words/s, in_qsize 30, out_qsize 3
  26. 2016-10-04 17:47:04,487 : INFO : PROGRESS: at 15.45% examples, 733252 words/s, in_qsize 29, out_qsize 0
  27. 2016-10-04 17:47:05,493 : INFO : PROGRESS: at 16.59% examples, 731323 words/s, in_qsize 31, out_qsize 1
  28. 2016-10-04 17:47:06,547 : INFO : PROGRESS: at 17.74% examples, 728241 words/s, in_qsize 29, out_qsize 4
  29. 2016-10-04 17:47:07,574 : INFO : PROGRESS: at 19.00% examples, 731231 words/s, in_qsize 32, out_qsize 2
  30. 2016-10-04 17:47:08,560 : INFO : PROGRESS: at 20.22% examples, 732649 words/s, in_qsize 29, out_qsize 0
  31. 2016-10-04 17:47:09,595 : INFO : PROGRESS: at 21.41% examples, 731891 words/s, in_qsize 31, out_qsize 1
  32. 2016-10-04 17:47:10,623 : INFO : PROGRESS: at 22.60% examples, 730945 words/s, in_qsize 28, out_qsize 4
  33. 2016-10-04 17:47:11,632 : INFO : PROGRESS: at 23.83% examples, 732614 words/s, in_qsize 30, out_qsize 3
  34. 2016-10-04 17:47:12,624 : INFO : PROGRESS: at 25.01% examples, 732840 words/s, in_qsize 31, out_qsize 0
  35. 2016-10-04 17:47:13,636 : INFO : PROGRESS: at 26.26% examples, 735019 words/s, in_qsize 31, out_qsize 0
  36. 2016-10-04 17:47:14,649 : INFO : PROGRESS: at 27.43% examples, 734959 words/s, in_qsize 31, out_qsize 0
  37. 2016-10-04 17:47:15,668 : INFO : PROGRESS: at 28.58% examples, 733902 words/s, in_qsize 32, out_qsize 0
  38. 2016-10-04 17:47:16,682 : INFO : PROGRESS: at 29.69% examples, 732120 words/s, in_qsize 31, out_qsize 4
  39. 2016-10-04 17:47:17,709 : INFO : PROGRESS: at 30.97% examples, 733963 words/s, in_qsize 31, out_qsize 0
  40. 2016-10-04 17:47:18,727 : INFO : PROGRESS: at 32.22% examples, 735482 words/s, in_qsize 29, out_qsize 0
  41. 2016-10-04 17:47:19,734 : INFO : PROGRESS: at 33.37% examples, 734576 words/s, in_qsize 32, out_qsize 0
  42. 2016-10-04 17:47:20,735 : INFO : PROGRESS: at 34.58% examples, 735437 words/s, in_qsize 32, out_qsize 0
  43. 2016-10-04 17:47:21,776 : INFO : PROGRESS: at 35.78% examples, 734396 words/s, in_qsize 30, out_qsize 2
  44. 2016-10-04 17:47:22,788 : INFO : PROGRESS: at 36.99% examples, 734990 words/s, in_qsize 31, out_qsize 2
  45. 2016-10-04 17:47:23,794 : INFO : PROGRESS: at 38.13% examples, 734017 words/s, in_qsize 31, out_qsize 0
  46. 2016-10-04 17:47:24,809 : INFO : PROGRESS: at 39.33% examples, 733968 words/s, in_qsize 32, out_qsize 0
  47. 2016-10-04 17:47:25,820 : INFO : PROGRESS: at 40.60% examples, 735458 words/s, in_qsize 31, out_qsize 0
  48. 2016-10-04 17:47:26,830 : INFO : PROGRESS: at 41.73% examples, 734095 words/s, in_qsize 31, out_qsize 0
  49. 2016-10-04 17:47:27,841 : INFO : PROGRESS: at 42.88% examples, 733521 words/s, in_qsize 29, out_qsize 3
  50. 2016-10-04 17:47:28,839 : INFO : PROGRESS: at 44.15% examples, 735077 words/s, in_qsize 25, out_qsize 0
  51. 2016-10-04 17:47:29,851 : INFO : PROGRESS: at 45.31% examples, 734870 words/s, in_qsize 30, out_qsize 0
  52. 2016-10-04 17:47:30,866 : INFO : PROGRESS: at 46.51% examples, 735385 words/s, in_qsize 24, out_qsize 1
  53. 2016-10-04 17:47:31,869 : INFO : PROGRESS: at 47.61% examples, 734076 words/s, in_qsize 29, out_qsize 4
  54. 2016-10-04 17:47:32,875 : INFO : PROGRESS: at 48.81% examples, 734331 words/s, in_qsize 29, out_qsize 0
  55. 2016-10-04 17:47:33,880 : INFO : PROGRESS: at 49.96% examples, 734036 words/s, in_qsize 31, out_qsize 0
  56. 2016-10-04 17:47:34,900 : INFO : PROGRESS: at 51.16% examples, 734204 words/s, in_qsize 28, out_qsize 3
  57. 2016-10-04 17:47:35,919 : INFO : PROGRESS: at 52.38% examples, 734554 words/s, in_qsize 32, out_qsize 1
  58. 2016-10-04 17:47:36,960 : INFO : PROGRESS: at 53.54% examples, 734277 words/s, in_qsize 28, out_qsize 3
  59. 2016-10-04 17:47:37,954 : INFO : PROGRESS: at 54.78% examples, 734888 words/s, in_qsize 31, out_qsize 3
  60. 2016-10-04 17:47:38,979 : INFO : PROGRESS: at 56.00% examples, 734850 words/s, in_qsize 27, out_qsize 7
  61. 2016-10-04 17:47:39,983 : INFO : PROGRESS: at 57.24% examples, 735531 words/s, in_qsize 29, out_qsize 3
  62. 2016-10-04 17:47:40,995 : INFO : PROGRESS: at 58.47% examples, 735891 words/s, in_qsize 29, out_qsize 1
  63. 2016-10-04 17:47:42,009 : INFO : PROGRESS: at 59.67% examples, 735953 words/s, in_qsize 31, out_qsize 0
  64. 2016-10-04 17:47:43,004 : INFO : PROGRESS: at 60.87% examples, 736214 words/s, in_qsize 31, out_qsize 0
  65. 2016-10-04 17:47:44,013 : INFO : PROGRESS: at 62.10% examples, 736671 words/s, in_qsize 27, out_qsize 1
  66. 2016-10-04 17:47:45,041 : INFO : PROGRESS: at 63.28% examples, 736105 words/s, in_qsize 30, out_qsize 3
  67. 2016-10-04 17:47:46,053 : INFO : PROGRESS: at 64.36% examples, 734979 words/s, in_qsize 31, out_qsize 8
  68. 2016-10-04 17:47:47,053 : INFO : PROGRESS: at 65.64% examples, 736172 words/s, in_qsize 32, out_qsize 0
  69. 2016-10-04 17:47:48,069 : INFO : PROGRESS: at 66.83% examples, 736263 words/s, in_qsize 29, out_qsize 2
  70. 2016-10-04 17:47:49,090 : INFO : PROGRESS: at 68.01% examples, 736256 words/s, in_qsize 29, out_qsize 1
  71. 2016-10-04 17:47:50,081 : INFO : PROGRESS: at 69.09% examples, 735169 words/s, in_qsize 31, out_qsize 0
  72. 2016-10-04 17:47:51,134 : INFO : PROGRESS: at 70.30% examples, 735037 words/s, in_qsize 32, out_qsize 3
  73. 2016-10-04 17:47:52,152 : INFO : PROGRESS: at 71.52% examples, 735505 words/s, in_qsize 28, out_qsize 5
  74. 2016-10-04 17:47:53,139 : INFO : PROGRESS: at 72.76% examples, 736054 words/s, in_qsize 32, out_qsize 0
  75. 2016-10-04 17:47:54,147 : INFO : PROGRESS: at 73.85% examples, 735154 words/s, in_qsize 32, out_qsize 0
  76. 2016-10-04 17:47:55,152 : INFO : PROGRESS: at 75.10% examples, 735713 words/s, in_qsize 31, out_qsize 0
  77. 2016-10-04 17:47:56,154 : INFO : PROGRESS: at 76.33% examples, 736086 words/s, in_qsize 26, out_qsize 0
  78. 2016-10-04 17:47:57,191 : INFO : PROGRESS: at 77.58% examples, 736364 words/s, in_qsize 32, out_qsize 1
  79. 2016-10-04 17:47:58,226 : INFO : PROGRESS: at 78.67% examples, 735123 words/s, in_qsize 30, out_qsize 3
  80. 2016-10-04 17:47:59,231 : INFO : PROGRESS: at 79.93% examples, 735788 words/s, in_qsize 30, out_qsize 2
  81. 2016-10-04 17:48:00,259 : INFO : PROGRESS: at 81.18% examples, 736015 words/s, in_qsize 32, out_qsize 1
  82. 2016-10-04 17:48:01,260 : INFO : PROGRESS: at 82.29% examples, 735365 words/s, in_qsize 32, out_qsize 0
  83. 2016-10-04 17:48:02,286 : INFO : PROGRESS: at 83.41% examples, 734608 words/s, in_qsize 30, out_qsize 3
  84. 2016-10-04 17:48:03,308 : INFO : PROGRESS: at 84.55% examples, 734154 words/s, in_qsize 29, out_qsize 2
  85. 2016-10-04 17:48:04,320 : INFO : PROGRESS: at 85.73% examples, 734110 words/s, in_qsize 28, out_qsize 3
  86. 2016-10-04 17:48:05,348 : INFO : PROGRESS: at 86.96% examples, 734438 words/s, in_qsize 30, out_qsize 1
  87. 2016-10-04 17:48:06,360 : INFO : PROGRESS: at 88.20% examples, 734932 words/s, in_qsize 27, out_qsize 0
  88. 2016-10-04 17:48:07,366 : INFO : PROGRESS: at 89.41% examples, 735145 words/s, in_qsize 32, out_qsize 0
  89. 2016-10-04 17:48:08,369 : INFO : PROGRESS: at 90.64% examples, 735636 words/s, in_qsize 30, out_qsize 0
  90. 2016-10-04 17:48:09,376 : INFO : PROGRESS: at 91.80% examples, 735508 words/s, in_qsize 31, out_qsize 0
  91. 2016-10-04 17:48:10,377 : INFO : PROGRESS: at 92.97% examples, 735425 words/s, in_qsize 30, out_qsize 1
  92. 2016-10-04 17:48:11,383 : INFO : PROGRESS: at 94.14% examples, 735413 words/s, in_qsize 32, out_qsize 1
  93. 2016-10-04 17:48:12,401 : INFO : PROGRESS: at 95.30% examples, 734952 words/s, in_qsize 31, out_qsize 1
  94. 2016-10-04 17:48:13,401 : INFO : PROGRESS: at 96.47% examples, 734838 words/s, in_qsize 31, out_qsize 0
  95. 2016-10-04 17:48:14,414 : INFO : PROGRESS: at 97.67% examples, 734919 words/s, in_qsize 31, out_qsize 0
  96. 2016-10-04 17:48:15,429 : INFO : PROGRESS: at 98.81% examples, 734553 words/s, in_qsize 24, out_qsize 3
  97. 2016-10-04 17:48:16,233 : INFO : worker thread finished; awaiting finish of 15 more threads
  98. 2016-10-04 17:48:16,236 : INFO : worker thread finished; awaiting finish of 14 more threads
  99. 2016-10-04 17:48:16,255 : INFO : worker thread finished; awaiting finish of 13 more threads
  100. 2016-10-04 17:48:16,279 : INFO : worker thread finished; awaiting finish of 12 more threads
  101. 2016-10-04 17:48:16,280 : INFO : worker thread finished; awaiting finish of 11 more threads
  102. 2016-10-04 17:48:16,280 : INFO : worker thread finished; awaiting finish of 10 more threads
  103. 2016-10-04 17:48:16,280 : INFO : worker thread finished; awaiting finish of 9 more threads
  104. 2016-10-04 17:48:16,283 : INFO : worker thread finished; awaiting finish of 8 more threads
  105. 2016-10-04 17:48:16,295 : INFO : worker thread finished; awaiting finish of 7 more threads
  106. 2016-10-04 17:48:16,302 : INFO : worker thread finished; awaiting finish of 6 more threads
  107. 2016-10-04 17:48:16,306 : INFO : worker thread finished; awaiting finish of 5 more threads
  108. 2016-10-04 17:48:16,308 : INFO : worker thread finished; awaiting finish of 4 more threads
  109. 2016-10-04 17:48:16,309 : INFO : worker thread finished; awaiting finish of 3 more threads
  110. 2016-10-04 17:48:16,310 : INFO : worker thread finished; awaiting finish of 2 more threads
  111. 2016-10-04 17:48:16,315 : INFO : worker thread finished; awaiting finish of 1 more threads
  112. 2016-10-04 17:48:16,321 : INFO : worker thread finished; awaiting finish of 0 more threads
  113. 2016-10-04 17:48:16,321 : INFO : training on 85026035 raw words (62532628 effective words) took 85.0s, 735445 effective words/s
  114. 2016-10-04 17:48:16,321 : INFO : precomputing L2-norms of word weight vectors
  115. 2016-10-04 17:48:16,678 : INFO : saving Word2Vec object under text8_data, separately None
  116. 2016-10-04 17:48:16,678 : INFO : not storing attribute syn0norm
  117. 2016-10-04 17:48:16,678 : INFO : not storing attribute cum_table
  118. 2016-10-04 17:48:17,373 : INFO : Removed 17 and 25 OOV words from document 1 and 2 (respectively).
  119. 2016-10-04 17:48:17,373 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  120. 2016-10-04 17:48:17,373 : INFO : built Dictionary(59 unique tokens: [u'heart', u'love', u'departed', u'deck', u'within']...) from 2 documents (total 75 corpus positions)
  121. Distance between the two poems is: 1.07660111163
  122. 2016-10-04 17:48:17,385 : INFO : Removed 0 and 1 OOV words from document 1 and 2 (respectively).
  123. 2016-10-04 17:48:17,385 : INFO : adding document #0 to Dictionary(0 unique tokens: [])
  124. 2016-10-04 17:48:17,385 : INFO : built Dictionary(7 unique tokens: [u'chicago', u'media', u'illinois', u'press', u'president']...) from 2 documents (total 7 corpus positions)
  125. Distance between the two sentences is: 1.10310476666
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!